# US 2017/0287472 A1 · Speech recognition apparatus and speech recognition method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0287472 A1
- Document ID: US20170287472A1
- Status: Published application
- Kind: A1
- Application number: 15/507,695
- Filed: 2014-12-18
- Published: 2017-10-05
- Earliest claimed: 2014-12-18
- Assignees: Mitsubishi Electric Corporation
- Applicants: Mitsubishi Electric Corporation
- Inventors: Isamu OGAWA; Toshiyuki HANAZAWA
- Web page: https://patentgrep.com/patent/US20170287472A1

## CPC

- G10L 15/18: using natural language modelling
- G06F 3/041: Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G10L 15/04: Segmentation; Word boundary detection
- G10L 15/24: Speech recognition using non-acoustical features
- G10L 15/25: using position of the lips, movement of the lips or face analysis
- G10L 2025/786: {Adaptive threshold}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An apparatus includes a lip image recognition unit 103 to recognize a user state from image data which is information other than speech; a non-speech section deciding unit 104 to decide from the recognized user state whether the user is talking; a speech section detection threshold learning unit 106 to set a first speech section detection threshold (SSDT) from speech data when decided not talking, and a second SSDT from the speech data after conversion by a speech input unit when decided talking; a speech section detecting unit 107 to detect a speech section indicating talking from the speech data using the thresholds set, wherein if it cannot detect the speech section using the second SSDT, it detects the speech section using the first SSDT; and a speech recognition unit 108 to recognize speech data in the speech section detected, and to output a recognition result.

## Claims

### Claim at position 1

1-6. (canceled)

### Claim 7

A speech recognition apparatus comprising:
a speech input unit to acquire collected speech and to convert the speech to speech data;
a non-speech information input unit to acquire information other than the speech;
a non-speech operation recognition unit to recognize a user state from the information other than the speech the non-speech information input unit acquires;
a non-speech section decider to decide whether the user is talking or not from the user state the non-speech operation recognition unit recognizes;
a threshold learning unit to set a first threshold from the speech data converted by the speech input unit when the non-speech section decider decides that the user is not talking, and to set a second threshold from the speech data converted by the speech input unit when the non-speech section decider decides that the user is talking;
a speech section detector to detect, using the threshold set by the threshold learning unit, a speech section indicating that the user is talking from the speech data converted by the speech input unit; and
a speech recognition unit to recognize the speech data in the speech section detected by the speech section detector, and to output a recognition result, wherein
the speech section detector detects the speech section by using the first threshold, if the speech section detector cannot detect the speech section by using the second threshold.

### Claim 8 (depends on 7)

The speech recognition apparatus according to claim 7, wherein
the non-speech information input unit acquires information about a position at which the user performs a touch input operation and acquires image data in which the user state is imaged, and
the non-speech operation recognition unit recognizes movement of the user's lips from the image data acquired by the non-speech information input unit, and
the non-speech section decider decides whether the user is talking or not from the information about the position acquired by the non-speech information input unit acquires and from the information indicating the movement of the lips the non-speech operation recognition unit recognizes.

### Claim 9 (depends on 7)

The speech recognition apparatus according to claim 7, wherein
the non-speech information input unit acquires information about a position at which the user performs a touch input operation, and
the non-speech operation recognition unit recognizes an operation state of operation input of the user from the information about the position the non-speech information input unit acquires and from transition information indicating the operation state of the user, which makes a transition in response to the touch input operation, and
the non-speech section decider decides whether the user is talking or not from the operation state the non-speech operation recognition unit recognizes and from the information about the position the non-speech information input unit acquires.

### Claim 10 (depends on 7)

The speech recognition apparatus according to claim 7, wherein
the non-speech information input unit acquires information about a position at which the user performs a touch input operation and acquires image data in which the user state is imaged, and
the non-speech operation recognition unit recognizes an operation state of operation input of the user from the information about the position the non-speech information input unit acquires and from transition information indicating the operation state of the user, which makes a transition in response to the touch input operation, and recognizes movement of the user's lips from the image data the non-speech information input unit acquires, and
the non-speech section decider decides whether the user is talking or not from the operation state the non-speech operation recognition unit recognizes, the information indicating the movement of the lips, and the information about the position the non-speech information input unit acquires.

### Claim 11 (depends on 7)

The speech recognition apparatus according to claim 7, wherein
the speech section detector counts time upon detection of a start point of the speech section, detects, in a case in which the speech section detector cannot detect an end point of the speech section even if the count value reaches a designated timeout point, an interval from the start point of the speech section to the timeout point, as the speech section using the second threshold, and detects the interval from the start point of the speech section to the timeout point, as the speech section of a correction candidate by using the first threshold, and
the speech recognition unit recognizes the speech data in the speech section detected by the speech section detector and outputs a recognition result, and recognizes the speech data in the speech section of the correction candidate and outputs a recognition result correction candidate.

### Claim 12

A speech recognition method comprising:
acquiring, by a speech input unit, collected speech and converting the speech to speech data;
acquiring, by a non-speech information input unit, information other than the speech;
recognizing, by a non-speech operation recognition unit, a user state from the information other than the speech;
deciding, by a non-speech section decider, whether the user is talking or not from the user state recognized:
setting, by a threshold learning unit, a first threshold from the speech data when decided that the user is not talking, and a second threshold when decided that the user is talking;
detecting, by a speech section detector, a speech section indicating that the user is talking from the speech data converted by the speech input unit by using the first threshold or the second, and detecting the speech section by using the first threshold when the speech section cannot be detected by using the second threshold; and
recognizing, by a speech recognition unit, speech data in the speech section detected, and outputting a recognition result.
