# US 12,001,533 B2 · Method and system for authenticating an occupant within an interior of a vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,001,533 B2
- Document ID: US12001533B2
- Status: Granted
- Kind: B2
- Application number: 17/392,083
- Filed: 2021-08-02
- Published: 2022-03-03
- Granted: 2024-06-04
- Earliest claimed: 2020-09-01
- Assignees: Aptiv Technologies AG
- Applicants: Aptiv Technologies AG
- Inventors: Alexander Barth; Douglas L. Welk
- Web page: https://patentgrep.com/patent/US12001533B2

## CPC

- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- B60R 25/25: using biometry
- G06F 18/2413: based on distances to training or reference patterns
- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V 40/172: {Classification, e.g. identification}
- G06V 40/40: Spoof detection, e.g. liveness detection
- G06V 40/45: {Detection of the body part being alive}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method is provided for authenticating an occupant within an interior of a vehicle. The vehicle includes a camera which is configured to monitor the interior of the vehicle, and a processing device being configured to process image or video data provided by the camera. Characteristic features are determined being visible via the camera, and authenticating image or video data are captured via the camera while an occupant is present within the interior of the vehicle. Via the processing device, face recognition and liveliness check are performed for the occupant based on the authenticating image or video data, and an identification degree of the characteristic features is determined within the authenticating image or video data. Based on the face recognition, the liveliness check and the identification degree it is determined whether an authentication of the occupant is to be accepted.

## Claims

### Claim 1

A method, comprising:
generating, with a camera, image or video data of an interior of a vehicle;
determining characteristic features that are visible via the camera; and
authenticating an occupant in the interior of the vehicle by:
capturing, from the image or video data generated by the camera, an authenticating image or video data that indicates a presence of the occupant within the interior of the vehicle;
performing, by a processor, based on the authenticating image or video data, a face recognition and a liveliness check for the occupant;
determining, within the authenticating image or video data, an identification degree of the characteristic features; and
determining whether to authenticate the occupant based on the face recognition, the liveliness check, and the identification degree of the characteristic features;
during capturing the authenticating image or video data, changing an illumination intensity within the interior of the vehicle; and
authenticating the occupant only if a detection intensity within the authenticating image or video data changes in accordance with the change of the illumination intensity.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the characteristic features comprises:
capturing, via the camera, background image or video data without any occupants present within the interior of the vehicle; and
selecting, from the background image or video data, the characteristic features.

### Claim 3 (depends on 2)

The method of claim 2, wherein the characteristic features are manually selected from the background image or video data during a calibration phase without any occupants present within the interior of the vehicle.

### Claim 4 (depends on 2)

The method of claim 2, wherein the characteristic features are automatically selected from the background image or video data by comparing the background image or video data with further image or video data captured from outside the interior of the vehicle.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining the identification degree of the characteristic features within the authenticating image or video data further comprises:
for each of the characteristic features, determining whether the respective characteristic feature is visible within the authentication image or video data;
determining a number of visible characteristic features for the authentication image or video data; and
authenticating the occupant only if the number of visible characteristic features exceeds a predefined threshold.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
capturing, via the camera, background image or video data without any occupants present within the interior of the vehicle;
capturing, via the camera, further image or video data from outside the interior of the vehicle; and
training an image classifier regarding the characteristic features based on the background image or video data and the further image or video data, and
wherein determining the identification degree of the characteristic features by employing the trained image classifier on the authenticating image or video data.

### Claim 7 (depends on 1)

The method of claim 1, wherein each of the characteristic features is related to an expected image section and to an expected position of the expected image section.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
detecting a plurality of positions for body key points of the occupant within the authenticating image or video data;
comparing distances between the positions for the body key points of the occupant with predefined distance ranges; and
authenticating the occupant only if at least a predetermined number of the distances falls into the predefined distance ranges.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
defining at least one authenticating region in which the face of the occupant is to be located for authentication within the interior of the vehicle;
determining, based on the authenticating image or video data, whether the face of the occupant is located within the authenticating region; and
authenticating the occupant only if the face of the occupant is determined to be within the authenticating region.

### Claim 10 (depends on 1)

The method of claim 1, wherein determining whether to authenticate the occupant is additionally based on a further authentication request.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
performing the further authentication request by:
defining a task that is to be performed by the occupant and that is suitable to be monitored by the camera;
capturing, via the camera, task image or video data while the occupant is expected to perform the task; and
determining, via the processor, whether to authenticate the occupant based on the task image or video data.

### Claim 12

A system, comprising:
a camera configured to monitor an interior of a vehicle by generating image or video data including by capturing authenticating image or video data while an occupant is present within the interior of the vehicle; and
a processor configured to:
determine characteristic features visible from the image or video data generated by the camera;
perform a face recognition and a liveliness check for the occupant based on the authenticating image or video data;
determine an identification degree of the characteristic features within the authenticating image or video data;
determine whether an authentication of the occupant is to be accepted based on the face recognition, the liveliness check and the identification degree of the characteristic features;
during capturing the authenticating image or video data, change an illumination intensity within the interior of the vehicle; and
authenticate the occupant only if a detection intensity within the authenticating image or video data changes in accordance with the change of the illumination intensity.

### Claim 13 (depends on 12)

The system of claim 12, wherein the processor is further configured to:
capture, via the camera, background image or video data without any occupants present within the interior of the vehicle;
capture, via the camera, further image or video data from outside the interior of the vehicle; and
train an image classifier regarding the characteristic features based on the background image or video data and the further image or video data, and
determine the identification degree of the characteristic features by employing the trained image classifier on the authenticating image or video data.

### Claim 14

A non-transitory computer readable medium comprising instructions that, when executed, cause hardware circuitry to:
generate, with a camera, image or video data of an interior of a vehicle;
determine characteristic features that are visible via the camera; and
authenticate an occupant in the interior of the vehicle by:
capturing, from the image or video data generated by the camera, an authenticating image or video data that indicates a presence of the occupant within the interior of the vehicle;
performing based on the authenticating image or video data, a face recognition and a liveliness check for the occupant;
determining, within the authenticating image or video data, an identification degree of the characteristic features;
determining whether to authenticate the occupant based on the face recognition, the liveliness check, and the identification degree of the characteristic features;
during capturing the authenticating image or video data, changing an illumination intensity within the interior of the vehicle; and
authenticating the occupant only if a detection intensity within the authenticating image or video data changes in accordance with the change of the illumination intensity.

### Claim 15 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the instructions, when executed, cause the hardware circuitry to determine the characteristic features by:
capturing, via the camera, background image or video data without any occupants present within the interior of the vehicle; and
selecting, from the background image or video data, the characteristic features.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the characteristic features are manually selected from the background image or video data during a calibration phase without any occupants present within the interior of the vehicle.

### Claim 17 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the characteristic features are automatically selected from the background image or video data by comparing the background image or video data with further image or video data captured from outside the interior of the vehicle.

### Claim 18 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the instructions, when executed, cause the hardware circuitry to determine the identification degree of the characteristic features within the authenticating image or video data by:
for each of the characteristic features, determining whether the respective characteristic feature is visible within the authentication image or video data;
determining a number of visible characteristic features for the authentication image or video data; and
authenticating the occupant only if the number of visible characteristic features exceeds a predefined threshold.

### Claim 19 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the instructions, when executed, further cause the hardware circuitry to:
capture, via the camera, background image or video data without any occupants present within the interior of the vehicle;
capture, via the camera, further image or video data from outside the interior of the vehicle; and
train an image classifier regarding the characteristic features based on the background image or video data and the further image or video data, and
determine the identification degree of the characteristic features by employing the trained image classifier on the authenticating image or video data.

### Claim 20

A method, comprising:
generating, with a camera, image or video data of an interior of a vehicle;
determining characteristic features that are visible via the camera; and
authenticating an occupant in the interior of the vehicle by:
capturing, from the image or video data generated by the camera, an authenticating image or video data that indicates a presence of the occupant within the interior of the vehicle;
performing, by a processor, based on the authenticating image or video data, a face recognition and a liveliness check for the occupant;
determining, within the authenticating image or video data, an identification degree of the characteristic features; and
determining whether to authenticate the occupant based on the face recognition, the liveliness check, and the identification degree of the characteristic features,
wherein determining the characteristic features comprises:
capturing, via the camera, background image or video data without any occupants present within the interior of the vehicle; and
selecting, from the background image or video data, the characteristic features, and
wherein the characteristic features are automatically selected from the background image or video data by comparing the background image or video data with further image or video data captured from outside the interior of the vehicle.

### Claim 21

A method, comprising:
generating, with a camera, image or video data of an interior of a vehicle;
determining characteristic features that are visible via the camera; and
authenticating an occupant in the interior of the vehicle by:
capturing, from the image or video data generated by the camera, an authenticating image or video data that indicates a presence of the occupant within the interior of the vehicle;
performing, by a processor, based on the authenticating image or video data, a face recognition and a liveliness check for the occupant;
determining, within the authenticating image or video data, an identification degree of the characteristic features; and
determining whether to authenticate the occupant based on the face recognition, the liveliness check, and the identification degree of the characteristic features capturing, via the camera, background image or video data without any occupants present within the interior of the vehicle;
capturing, via the camera, further image or video data from outside the interior of the vehicle; and
training an image classifier regarding the characteristic features based on the background image or video data and the further image or video data, and
wherein determining the identification degree of the characteristic features by employing the trained image classifier on the authenticating image or video data.

### Claim 22

A system, comprising:
a camera configured to monitor an interior of a vehicle by generating image or video data including by capturing authenticating image or video data while an occupant is present within the interior of the vehicle; and
a processor configured to:
determine characteristic features visible from the image or video data generated by the camera;
perform a face recognition and a liveliness check for the occupant based on the authenticating image or video data;
determine an identification degree of the characteristic features within the authenticating image or video data;
determine whether an authentication of the occupant is to be accepted based on the face recognition, the liveliness check and the identification degree of the characteristic features;
wherein the processor is further configured to:
capture, via the camera, background image or video data without any occupants present within the interior of the vehicle;
capture, via the camera, further image or video data from outside the interior of the vehicle; and
train an image classifier regarding the characteristic features based on the background image or video data and the further image or video data, and
determine the identification degree of the characteristic features by employing the trained image classifier on the authenticating image or video data.

### Claim 23

A non-transitory computer readable medium comprising instructions that, when executed, cause hardware circuitry to:
generate, with a camera, image or video data of an interior of a vehicle;
determine characteristic features that are visible via the camera; and
authenticate an occupant in the interior of the vehicle by:
capturing, from the image or video data generated by the camera, an authenticating image or video data that indicates a presence of the occupant within the interior of the vehicle;
performing based on the authenticating image or video data, a face recognition and a liveliness check for the occupant;
determining, within the authenticating image or video data, an identification degree of the characteristic features;
determining whether to authenticate the occupant based on the face recognition, the liveliness check, and the identification degree of the characteristic features,
wherein the instructions, when executed, cause the hardware circuitry to determine the characteristic features by:
capturing, via the camera, background image or video data without any occupants present within the interior of the vehicle; and
selecting, from the background image or video data, the characteristic features, and
wherein the characteristic features are automatically selected from the background image or video data by comparing the background image or video data with further image or video data captured from outside the interior of the vehicle.

### Claim 24

A non-transitory computer readable medium comprising instructions that, when executed, cause hardware circuitry to:
generate, with a camera, image or video data of an interior of a vehicle;
determine characteristic features that are visible via the camera; and
authenticate an occupant in the interior of the vehicle by:
capturing, from the image or video data generated by the camera, an authenticating image or video data that indicates a presence of the occupant within the interior of the vehicle;
performing based on the authenticating image or video data, a face recognition and a liveliness check for the occupant;
determining, within the authenticating image or video data, an identification degree of the characteristic features;
determining whether to authenticate the occupant based on the face recognition, the liveliness check, and the identification degree of the characteristic features,
wherein the instructions, when executed, further cause the hardware circuitry to:
capture, via the camera, background image or video data without any occupants present within the interior of the vehicle;
capture, via the camera, further image or video data from outside the interior of the vehicle; and
train an image classifier regarding the characteristic features based on the background image or video data and the further image or video data, and
determine the identification degree of the characteristic features by employing the trained image classifier on the authenticating image or video data.