# US 2021/0174790 A1 · Vehicle operation assistance device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174790 A1
- Document ID: US20210174790A1
- Status: Published application
- Kind: A1
- Application number: 16/764,705
- Filed: 2017-11-17
- Published: 2021-06-10
- Earliest claimed: 2017-11-17
- Assignees: Nissan Motor Co., Ltd.
- Applicants: Nissan Motor Co., Ltd.
- Inventors: Jo Nishiyama; Hirofumi Inoue; Takehito Teraguchi; Jun Kasai
- Web page: https://patentgrep.com/patent/US20210174790A1

## CPC

- G10L 15/1822: {Parsing for meaning understanding}
- G10L 15/183: using context dependencies, e.g. language models
- G10L 17/22: Interactive procedures; Man-machine interfaces
- G10L 25/51: for comparison or discrimination
- B60R 16/037: for occupant comfort {, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel}
- B60R 16/0373: {Voice control (in general G10L)}
- G01C 21/3617: {using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement}
- G10L 2015/088: {Word spotting}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)

## Abstract

An operation assistance device for vehicles includes a human relationship analysis unit, a human relationship storage unit, and an assistance information determination unit. The operation assistance device obtains recommendation information suitable for an occupant composition including human relationships and provides the recommendation information for assisting a vehicle operation. The analysis unit acquires conversation voice data of a plurality of persons to specify speakers, analyzes the acquired data for each specified speaker to extract a predetermined keyword, specifies a speech pattern of each speaker based on the keyword, specifies conversation content based on the acquired data, and analyzes and quantifies direct human relationships among the persons from the specified speech pattern and conversation content. The storage unit preliminarily accumulates the direct human relationships. The determination unit specifies a plurality of occupants on board a vehicle and determines, based on the accumulated direct human relationships, assistance information for the vehicle operation.

## Claims

### Claim at position 1

1-7. (canceled)

### Claim 8

An operation assistance device for vehicles, comprising:
a human relationship analysis unit;
a human relationship storage unit; and
an assistance information determination unit,
the human relationship analysis unit operating to acquire direct human relationships among a plurality of persons who can be occupants, the direct human relationships being analyzed from conversation content of each of speakers included in conversation voice data of the plurality of persons,
the human relationship storage unit operating to accumulate the direct human relationships among the plurality of persons who can be the occupants,
the assistance information determination unit operating to specify a plurality of occupants on board a vehicle and determine, on a basis of the direct human relationships accumulated in the human relationship storage unit, assistance information for a vehicle operation determined to be recommended in accordance with human relationships among the plurality of occupants,
wherein the conversation voice data of the plurality of persons who can be the occupants is detected by a terminal capable of collecting voices even when the persons are not on board the vehicle and is transmitted to the human relationship analysis unit.

### Claim 9 (depends on 8)

The operation assistance device for vehicles according to claim 8, wherein the human relationship analysis unit operates to:
acquire the conversation voice data of the plurality of persons who can be the occupants, to specify the speakers;
analyze the acquired conversation voice data for each of the specified speakers to extract a predetermined keyword;
specify a speech pattern of each of the speakers on a basis of the keyword for each of the speakers;
specify conversation content on a basis of the conversation voice data of the plurality of persons; and
analyze and quantify the direct human relationships among the plurality of persons from the specified speech pattern and the specified conversation content.

### Claim 10 (depends on 9)

The operation assistance device for vehicles according to claim 9, wherein
the human relationship analysis unit operates to estimate and quantify, on a basis of the quantified direct human relationships, indirect human relationships among unanalyzed persons of the persons stored in the human relationship storage unit,
the human relationship storage unit operates to further accumulate the indirect human relationships, and
the assistance information determination unit operates to determine, on a basis of the direct human relationships and indirect human relationships accumulated in the human relationship storage unit, the assistance information for the vehicle operation determined to be recommended in accordance with the human relationships among the plurality of occupants.

### Claim 11 (depends on 10)

The operation assistance device for vehicles according to claim 10, wherein,
when estimating and quantifying, on the basis of the quantified direct human relationships, the indirect human relationships among the unanalyzed persons of the persons stored in the human relationship storage unit, the human relationship analysis unit operates to:
statistically process relationships among a plurality of human relationships V1, V2, and V3 using an already quantified direct human relationship V1 between a first person and a second person, an already quantified direct human relationship V2 between the second person and a third person, and an already quantified direct human relationship V3 between the third person and the first person; and
estimate an unanalyzed human relationship V3′ from the already quantified remaining human relationships V1 and V2 and the statistically processed relationships among the human relationships V1, V2, and V3.

### Claim 12 (depends on 8)

The operation assistance device for vehicles according to claim 8, further comprising:
a vehicle information learning unit; and
an operation tendency storage unit,
the vehicle information learning unit operating to:
specify the plurality of occupants on board the vehicle;
extract the human relationships regarding the plurality of occupants from the human relationship storage unit;
detect operation information regarding the vehicle operation; and
associate the extracted human relationships with the detected operation information,
the operation tendency storage unit operating to accumulate the associated human relationships and operation information,
the assistance information determination unit operating to determine the operation information associated with the human relationships accumulated in the operation tendency storage unit as the assistance information for the vehicle operation determined to be recommended.

### Claim 13 (depends on 8)

The operation assistance device for vehicles according to claim 8, wherein,
when acquiring the conversation voice data of the plurality of persons to specify the speakers, the human relationship analysis unit operates to:
detect speech positions of a plurality of conversation voice data;
extract, from the plurality of conversation voice data, a conversation voice data set in which the speech positions are not more than a predetermined distance;
group the speakers of the conversation voice data set into a conversation group; and
specify the speakers who belong to the conversation group.

### Claim 14 (depends on 8)

The operation assistance device for vehicles according to claim 8, wherein,
when acquiring the conversation voice data of the plurality of persons to specify the speakers, the human relationship analysis unit operates to:
detect speech periods of a plurality of conversation voice data;
extract, from the plurality of conversation voice data, a conversation voice data set in which the speech periods do not overlap for a predetermined time or more;
group the speakers of the conversation voice data set into a conversation group; and
specify the speakers who belong to the conversation group.

### Claim 15 (depends on 8)

The operation assistance device for vehicles according to claim 8, wherein the terminal operates to transmit ID and positional information of the terminal to the human relationship analysis unit.

### Claim 16 (depends on 15)

The operation assistance device for vehicles according to claim 15, wherein the human relationship analysis unit operates to acquire the direct human relationships among the plurality of persons, wherein the direct human relationships are analyzed on a basis of the ID, positional information, and conversation voice data of the terminal.

### Claim 17 (depends on 15)

The operation assistance device for vehicles according to claim 15, wherein the human relationship analysis unit operates to acquire the direct human relationships among the plurality of persons, wherein the direct human relationships are analyzed on a basis of distances between a plurality of the terminals.

### Claim 18 (depends on 8)

The operation assistance device for vehicles according to claim 8, wherein the direct human relationships accumulated in the human relationship storage unit include a predetermined human relationship and its probability value.

### Claim 19

An operation assistance method for vehicles for assisting a vehicle operation using an operation assistance device for vehicles, the operation assistance device comprising a human relationship analysis unit, a human relationship storage unit, and an assistance information determination unit, the operation assistance method comprising:
operating the human relationship analysis unit to acquire direct human relationships among a plurality of persons who can be occupants, the direct human relationships being analyzed from conversation content of each of speakers included in conversation voice data of the plurality of persons;
operating the human relationship storage unit to accumulate the direct human relationships among the plurality of persons who can be the occupants; and
operating the assistance information determination unit to specify a plurality of occupants on board a vehicle and determine, on a basis of the direct human relationships accumulated in the human relationship storage unit, assistance information for the vehicle operation determined to be recommended in accordance with human relationships among the plurality of occupants,
wherein the conversation voice data of the plurality of persons who can be the occupants is detected by a terminal capable of collecting voices even when the persons are not on board the vehicle and is transmitted to the human relationship analysis unit.
