# US 2021/0333869 A1 · Vehicle control device and vehicle control method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0333869 A1
- Document ID: US20210333869A1
- Status: Published application
- Kind: A1
- Application number: 16/483,158
- Filed: 2018-11-30
- Published: 2021-10-28
- Earliest claimed: 2018-11-30
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Jaehoon CHO; Doyun PARK; Joohyeon OH; Dongkyu LEE; Heejeong HEO
- Web page: https://patentgrep.com/patent/US20210333869A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/22: Display screens
- B60K 35/25: using haptic output
- B60K 35/26: using acoustic output
- B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
- B60K 35/81: for controlling displays
- B60R 11/02: for radio sets, television sets, telephones, or the like; Arrangement of controls thereof
- B60R 21/015: including means for detecting the presence or position of passengers, passenger seats or child seats {, and the related safety parameters therefor, e.g. speed or timing of airbag inflation in relation to occupant position or seat belt use}
- B60W 40/08: related to drivers or passengers
- B60W 50/08: Interaction between the driver and the control system
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- B60W 50/16: Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- H04R 1/08: Mouthpieces; {Microphones;} Attachments therefor
- B60K 35/65: Instruments specially adapted for specific vehicle types or users, e.g. for left- or right-hand drive
- B60K 2360/1438: Touch screens
- B60K 2360/146: Instrument input by gesture
- B60K 2360/1464: 3D-gesture
- B60K 2360/148: Instrument input by voice
- B60K 2360/149: Instrument input by detecting viewing direction not otherwise provided for
- B60K 2360/176: Camera images
- B60K 2360/21: using cameras
- B60K 2360/741: Instruments adapted for user detection
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- H04R 2499/13: Acoustic transducers and sound field adaptation in vehicles
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Disclosed herein is a control method. The control method includes monitoring a gaze of an occupant present inside a vehicle through a camera, receiving a control command from the occupant through an input unit, determining a control target on the basis of a result of monitoring the gaze of the occupant and the control command, and controlling the control target on the basis of the control command.

## Claims

### Claim 1

A control method comprising:
monitoring a gaze of an occupant present inside a vehicle through a camera;
receiving a control command from the occupant through an input unit;
determining a control target on the basis of a result of monitoring the gaze of the occupant and the control command; and
controlling the control target on the basis of the control command.

### Claim 2 (depends on 1)

The control method of claim 1, wherein
the determining of a control target comprises:
searching for a plurality of control targets on the basis of the result of monitoring of the gaze of the occupant; and
specifying one of the plurality of control targets on the basis of the control command from the occupant.

### Claim 3 (depends on 1)

The control method of claim 1, further comprising:
continuing to monitor the gaze of the occupant when the control target is determined.

### Claim 4 (depends on 3)

The control method of claim 3, wherein,
once the control target is determined, the control target is controlled regardless of the result of monitoring of the gaze of the occupant.

### Claim 5 (depends on 1)

The control method of claim 1, wherein
the input unit is a microphone installed inside the vehicle,
wherein the control method further comprising:
receiving an utterance of the occupant through the microphone.

### Claim 6 (depends on 5)

The control method of claim 5, further comprising:
lowering a weight of the result of monitoring of the gaze of the occupant and raising a weight of the utterance of the occupant received through the microphone when the control target is controlled on the basis of the control command.

### Claim 7 (depends on 1)

The control method of claim 1, wherein
the input unit is the camera,
wherein the control method further comprising:
receiving a touch input or a gesture input of the occupant as the control command through the camera,
wherein the determining of a control target comprises: determining the control target on the basis of at least one of the touch input or the gesture input of the occupant and the result of monitoring of the gaze of the occupant.

### Claim 8 (depends on 1)

The control method of claim 1, wherein
the input unit comprises the camera and a microphone installed in the vehicle,
wherein the control method further comprising:
receiving an utterance of the occupant as a first control command through the microphone;
receiving a touch input or a gesture input of the occupant as a second control command through the camera; and
determining the control target on the basis of the first control command and the second control command if accuracy of the result of monitoring of the gaze of the occupant is equal to or less than a reference value.

### Claim 9 (depends on 1)

The control method of claim 1, further comprising:
providing feedback to the occupant through a feedback unit installed inside the vehicle and including a reference region.

### Claim 10 (depends on 9)

The control method of claim 9, further comprising:
rotating the feedback unit such that the reference region faces the control target when the control target is determined.

### Claim 11 (depends on 9)

The control method of claim 9, further comprising:
rotating the feedback unit such that the reference region faces the occupant when controlling of the control target is completed.

### Claim 12 (depends on 9)

The control method of claim 9, wherein
the occupant comprises a first occupant and a second occupant, and
wherein the control method further comprising:
rotating the feedback unit such that the reference region faces the second occupant when the control command is received from the second occupant.

### Claim 13 (depends on 1)

The control method of claim 1, further comprising:
displaying an image related to the control target on a display unit inside the vehicle when the control target is determined.

### Claim 14 (depends on 1)

The control method of claim 1, further comprising:
displaying an image related to the control target on the display unit of the vehicle when the control target is controlled.

### Claim 15 (depends on 1)

The control method of claim 1, further comprising:
when the control target is determined as a display unit inside the vehicle and the result of monitoring of the gaze of the occupant is a first region of the display unit, displaying an image in the first region on the basis of the control command.

### Claim 16 (depends on 15)

The control method of claim 15, further comprising:
monitoring a gaze of the occupant, after displaying the image in the first region, to obtain a second monitoring result;
receiving a second control command from the occupant; and
displaying an image in a second region of the display unit on the basis of the second control command when a second control target is determined as the second region on the basis of the second monitoring result and the second control command.

### Claim 17 (depends on 1)

The control method of claim 1, further comprising:
providing feedback to the occupant through a sound output unit installed inside the vehicle.

### Claim 18

A control device comprising:
an interface unit connected to a camera for monitoring a gaze of an occupant and an input unit receiving a control command from the occupant; and
a processor exchanging information with the camera and the input unit through the interface unit,
wherein the processor determines a control target on the basis of a result of monitoring the gaze of the occupant and the control command and controls the control target on the basis of the control command.

### Claim 19 (depends on 18)

The control device of claim 18, wherein
the processor searches for a plurality of control targets on the basis of the result of monitoring of the gaze of the occupant and specifies one of the plurality of control targets on the basis of the control command of the occupant.

### Claim 20 (depends on 18)

The control device of claim 18, wherein
when the control target is specified, the processor continues monitoring the gaze of the occupant.
