# US 2021/0334564 A1 · Method for monitoring an occupant and a device therefor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334564 A1
- Document ID: US20210334564A1
- Status: Published application
- Kind: A1
- Application number: 16/487,822
- Filed: 2018-11-21
- Published: 2021-10-28
- Earliest claimed: 2018-11-21
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Minsick PARK
- Web page: https://patentgrep.com/patent/US20210334564A1

## CPC

- B60R 21/01552: {detecting position of specific human body parts, e.g. face, eyes or hands}
- B60K 28/04: responsive to presence or absence of the driver, e.g. to weight or lack thereof
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/22: Display screens
- B60K 35/25: using haptic output
- B60K 35/265: {Voice}
- B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
- B60K 35/85: Arrangements for transferring vehicle- or driver-related data
- B60Q 9/00: Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 {, e.g. haptic signalling}
- B60R 21/015: including means for detecting the presence or position of passengers, passenger seats or child seats {, and the related safety parameters therefor, e.g. speed or timing of airbag inflation in relation to occupant position or seat belt use}
- B60R 21/01538: {for image processing, e.g. cameras or sensor arrays}
- B60R 21/01556: {Child-seat detection systems}
- B60R 21/16: Inflatable occupant restraints or confinements designed to inflate upon impact or impending impact, e.g. air bags ({protective garments with automatically inflatable shock-absorbing means A41D13/018; } connection of valves to inflatable elastic bodies B60C29/00)
- B60W 40/08: related to drivers or passengers
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- G06F 18/24: Classification techniques
- G06T 7/11: Region-based segmentation
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88)
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V 20/593: {Recognising seat occupancy}
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V 40/161: {Detection; Localisation; Normalisation}
- B60R 2021/0032: {Position of passenger}
- B60R 2021/01204: {Actuation parameters of safety arrangents}
- G06F 18/2413: based on distances to training or reference patterns
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/30268: Vehicle interior
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present invention relates to an occupant monitoring method and apparatus therefor. A monitoring apparatus according to one aspect of the present invention may include a camera obtaining an image in a vehicle and a processor processing the image. Moreover, the processor may include a detecting module separating each region in which an object exists from the image, a classifying module classifying the object existing in the each separated region and a recognizing module recognizing a pose of an occupant if the object corresponds to the occupant.

## Claims

### Claim 1

An apparatus for monitoring an occupant, the apparatus comprising:
a camera obtaining an image in a vehicle; and
a processor processing the image,
wherein the processor comprises a detecting module separating each region in which an object exists from the image, a classifying module classifying the object existing in the each separated region and a recognizing module recognizing a pose of the occupant if the object corresponds to the occupant.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein learning data of an object model is defined in advance based on deep learning and wherein the detecting module separates the object existing region using the learning data.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the learning data is defined in advance based on vehicle 3D information on information of the vehicle and object 3D rendering information inputted from a user.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein the detecting module matches up the learning data defined in advance with the obtained image in the vehicle, separates a seat region of the vehicle based on the matching, and separates the object existing region from the separated seat region.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein if the object is a Chair Restriction Seat (CRS), the classifying module detects an installed state of the CRS through image processing and detects a volume of the occupant in the CRS.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein if the CRS is installed in a rear direction of the vehicle and the detected volume of the occupant is in a preset range, the processor controls an airbag to be turned off.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein if a first object corresponding to the occupant and a second object corresponding to a thing exist in the separated region, the classifying module detects a region in which the first and second objects overlap each other.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein if the overlapped region is smaller than a preset first threshold, the recognizing module recognizes a pose of the first object using skeleton tracking and wherein if the pose of the first object corresponds to a preset pose, the processor outputs a warning or controls on/off of an airbag.

### Claim 9 (depends on 7)

The apparatus of claim 7, wherein if the overlapped region is equal to or greater than the preset first threshold, the recognizing module detects an inclination of the first object based on a face position of the first object and a position of a center point of the second object and wherein if the inclination is greater than a preset second threshold, the processor outputs a warning or controls on/off of an airbag.

### Claim 10 (depends on 7)

The apparatus of claim 7, wherein the classifying module extracts a context of the first object based on a position of the second object.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the processor controls at least one of the airbag, a display and/or an audio based on the extracted context.

### Claim 12

A method of monitoring an occupant, the method comprising:
obtaining an image in a vehicle through a camera;
separating each region in which an object exists from the image;
classifying the object existing in the each separated region; and
recognizing a pose of the occupant if the object corresponds to the occupant.
