# US 10,870,436 B2 · Operation assistance system and operation assistance method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,870,436 B2
- Document ID: US10870436B2
- Status: Granted
- Kind: B2
- Application number: 16/519,549
- Filed: 2019-07-23
- Published: 2020-01-30
- Granted: 2020-12-22
- Earliest claimed: 2018-07-24
- Assignees: Honda Motor Co., Ltd.
- Applicants: Honda Motor Co., Ltd.
- Inventors: Kota Saito; Yoshihiro Matsugi; Yuki Matsuura; Seungho Choi; Yoshikazu Matsuo; Hiromi Sato
- Web page: https://patentgrep.com/patent/US10870436B2

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/22: Display screens
- B60W 50/10: Interpretation of driver requests or demands
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- B60K 2360/1464: 3D-gesture
- B60W 2050/143: {Alarm means}
- B60W 2050/146: {Display means}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An operation assistance system includes: a motion recognition unit that recognizes predetermined motion of a hand of a vehicle occupant; a position recognition unit that recognizes a first relative position corresponding to a relative position of the hand of the vehicle occupant with respect to a first vehicle; a specifying unit that specifies a second on-vehicle device of a second vehicle that is operated by using an operation unit of the second vehicle provided at a second relative position that is a relative position with respect to the second vehicle and corresponds to the first relative position; and a control unit that assists an operation for a first on-vehicle device of the first vehicle, the first on-vehicle device including a function common with that of the second on-vehicle device.

## Claims

### Claim 1

An operation assistance system configured to assist an operation for a first on-vehicle device of a first vehicle, the operation assistance system comprising:
a motion recognition unit configured to recognize a predetermined motion of a hand of a vehicle occupant in the first vehicle;
a position recognition unit configured to recognize a first relative position corresponding to a relative position of the hand of the vehicle occupant with respect to the first vehicle;
a recording unit configured to record a relative position of an operation unit of a second vehicle with respect to the second vehicle, and a second on-vehicle device of the second vehicle that is operated by using the operation unit, while associating the relative position with the second on-vehicle device;
a specifying unit configured to specify the second on-vehicle device of the second vehicle that is operated by using the operation unit of the second vehicle provided at a second relative position on a basis of a record in the recording unit, the second relative position being a relative position with respect to the second vehicle and corresponding to the first relative position; and
a control unit configured to assist the operation for the first on-vehicle device of the first vehicle, the first on-vehicle device including a function common with that of the second on-vehicle device.

### Claim 2 (depends on 1)

The operation assistance system according to claim 1, wherein:
the motion recognition unit is configured to recognize as the predetermined motion, a motion that is identified as a state where the vehicle occupant is confused about the operation; and
if the motion recognition unit recognizes the motion identified as the state where the vehicle occupant is confused about the operation, the control unit is configured to assist the operation for the first on-vehicle device.

### Claim 3 (depends on 1)

The operation assistance system according to claim 1, wherein the vehicle occupant has previously occupied the second vehicle, and the recording unit is configured to record the relative position of the operation unit of the second vehicle with respect to the second vehicle, and the second on-vehicle device of the second vehicle that is operated by using the operation unit, while associating the relative position with the second on-vehicle device.

### Claim 4 (depends on 1)

The operation assistance system according to claim 1, wherein the vehicle occupant has previously occupied the second vehicle, the recording unit is configured to record vehicle type information corresponding to at least one piece of manufacturer information that expresses a manufacturer of the second vehicle, manufacturing country information that expresses a manufacturing country of the second vehicle, and vehicle model information that expresses a model of the second vehicle, while associating the vehicle type information with the vehicle occupant; and
if at least a part of vehicle type information of the first vehicle does not match the vehicle type information of the second vehicle, the control unit is configured to assist the operation for the first on-vehicle device.

### Claim 5 (depends on 1)

The operation assistance system according to claim 1, wherein the first on-vehicle device and the second on-vehicle device are at least one of an air conditioner device, a hazard lamp device, and a wiper device.

### Claim 6

An operation assistance method for assisting an operation for a first on-vehicle device of a first vehicle, the operation assistance method comprising:
a step of, by a recording unit, recording a relative position of an operation unit of a second vehicle with respect to the second vehicle, and a second on-vehicle device of the second vehicle that is operated by using the operation unit, while associating the relative position with the second on-vehicle device;
a motion recognition step of, by a calculation unit, recognizing a predetermined motion of a hand of a vehicle occupant in the first vehicle;
a position recognition step of, by the calculation unit, recognizing a first relative position corresponding to a relative position of the hand of the vehicle occupant with respect to the first vehicle;
a specifying step of, by the calculation unit, specifying the second on-vehicle device of the second vehicle that is operated by using the operation unit of the second vehicle provided at a second relative position, on a basis of a record in the recording unit, the second relative position being a relative position with respect to the second vehicle and corresponding to the first relative position; and
a control step of, by the calculation unit, assisting the operation for the first on-vehicle device of the first vehicle, the first on-vehicle device including a function common with that of the second on-vehicle device.