# US 2022/0036877 A1 · Speech recognition device, speech recognition system, and speech recognition method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036877 A1
- Document ID: US20220036877A1
- Status: Published application
- Kind: A1
- Application number: 17/278,725
- Filed: 2018-10-15
- Published: 2022-02-03
- Earliest claimed: 2018-10-15
- Assignees: Mitsubishi Electric Corporation
- Applicants: Mitsubishi Electric Corporation
- Inventors: Naoya BABA; Yusuke KOJI
- Web page: https://patentgrep.com/patent/US20220036877A1

## CPC

- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/25: using position of the lips, movement of the lips or face analysis
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G10L 15/24: Speech recognition using non-acoustical features
- G10L 21/0272: Voice signal separating
- G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10)
- G10L 2015/226: {using non-speech characteristics}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A speech signal processing unit individually separates uttered speech of a plurality of passengers each seated in one of a plurality of speech recognition target seats in a vehicle. A speech recognition unit performs speech recognition on uttered speech of each of the passengers separated by the speech signal processing unit and calculates a speech recognition score. A score-using determining unit determines a speech recognition result of which of the passengers is to be used from among speech recognition results for the passengers, using the speech recognition score of each of the passengers.

## Claims

### Claim at position 1

-12. (canceled)

### Claim 13

A speech recognition device comprising:
processing circuitry to
individually separate uttered speech of a plurality of passengers each seated in one of a plurality of speech recognition target seats in a vehicle;
perform speech recognition on the separated uttered speech of each of the passengers calculate a speech recognition score;
comprehend intention of utterance of each of the passengers and calculate an intention comprehension score using a speech recognition result of each of the passengers; and
determine an intention comprehension result of which of the passengers is to be used from among intention comprehension results for the respective passengers, using at least one of the speech recognition score or the intention comprehension score of each of the passengers;
calculate a face feature amount for each of the passengers using an image capturing the plurality of passengers; and
determine whether or not there is utterance for each of the passengers using the face feature amount from a start time to an end time of the uttered speech of each of the passengers,
wherein, in a case where there are identical intention comprehension results that correspond to two or more passengers determined to be speaking, the processing circuitry determines an intention comprehension result of which of the passengers is adopted from among the intention comprehension results that correspond to the two or more passengers, using at least one of the speech recognition score or the intention comprehension score of each of the two or more passengers.

### Claim 14 (depends on 13)

The speech recognition device according to claim 13,
wherein the processing circuitry determines an utterance period for each of the passengers using the face feature amount of each of the passengers,
the processing circuitry does not perform speech recognition on uttered speech of a passenger determined to have no utterance period, and
the processing circuitry does not comprehend intention of utterance of the passenger determined to have no utterance period.

### Claim 15 (depends on 13)

The speech recognition device according to claim 13, further comprising:
a dialogue management database to define correspondence between intention comprehension results and functions to be executed,
wherein the processing circuitry determines a function that corresponds to an intention comprehension result adopted by referring to the dialogue management database.

### Claim 16 (depends on 13)

The speech recognition device according to claim 13,
wherein the processing circuitry calculates a determination score indicating reliability of determination as to whether or not there is utterance for each of the passengers, and
in a case where there are identical intention comprehension results that correspond to two or more passengers determined to be speaking, the processing circuitry determines whether or not to adopt the intention comprehension results using the determination score in addition to at least one of the speech recognition score or the intention comprehension score of each of the two or more passengers.

### Claim 17

A speech recognition system comprising:
processing circuitry to
individually separate uttered speech of a plurality of passengers each seated in one of a plurality of speech recognition target seats in a vehicle;
perform speech recognition on the separated uttered speech of each of the passengers and calculate a speech recognition score;
comprehend intention of utterance of each of the passengers and calculating an intention comprehension score using a speech recognition result of each of the passengers; and
determine an intention comprehension result of which of the passengers is to be used from among intention comprehension results for the respective passengers, using at least one of the speech recognition score or the intention comprehension score of each of the passengers;
calculate a face feature amount for each of the passengers using an image capturing the plurality of passengers; and
determine whether or not there is utterance for each of the passengers using the face feature amount from a start time to an end time of the uttered speech of each of the passengers,
wherein, in a case where there are identical intention comprehension results that correspond to two or more passengers determined to be speaking, the processing circuitry determines an intention comprehension result of which of the passengers is adopted from among the intention comprehension results that correspond to the two or more passengers, using at least one of the speech recognition score or the intention comprehension score of each of the two or more passengers.

### Claim 18

A speech recognition method comprising:
individually separating uttered speech of a plurality of passengers each seated in one of a plurality of speech recognition target seats in a vehicle;
performing speech recognition on the separated uttered speech of each of the passengers and calculating a speech recognition score;
comprehending intention of utterance of each of the passengers and calculating an intention comprehension score using a speech recognition result of each of the passengers;
determining an intention comprehension result of which of the passengers is to be used from among intention comprehension results for the respective passengers, using at least one of the speech recognition score or the intention comprehension score of each of the passengers;
calculating a face feature amount for each of the passengers using an image capturing the plurality of passengers;
determining whether or not there is utterance for each of the passengers using the face feature amount from a start time to an end time of the uttered speech of each of the passengers; and
determining an intention comprehension result of which of the passengers is adopted from among the intention comprehension results that correspond to the two or more passengers, using at least one of the speech recognition score or the intention comprehension score of each of the two or more passengers in a case where there are identical intention comprehension results that correspond to two or more passengers determined to be speaking.
