Back to results
US 9,286,897 B2Granted
Speech recognizer with multi-directional decoding
AssigneeAmazon Technologies, Inc.
InventorsMichael Maximilian Emanuel Bisani, Nikko Strom, Bjorn Hoffmeister, Ryan Paul Thomas
Life of this patent
Estimating the term…
Abstract
In an automatic speech recognition (ASR) processing system, ASR processing may be configured to process speech based on multiple channels of audio received from a beamformer. The ASR processing system may include a microphone array and the beamformer to output multiple channels of audio such that each channel isolates audio in a particular direction. The multichannel audio signals may include spoken utterances/speech from one or more speakers as well as undesired audio, such as noise from a household appliance. The ASR device may simultaneously perform speech recognition on the multi-channel audio to provide more accurate speech recognition results.
Claims19
A method for performing speech recognition, the method comprising:
- receiving a multiple-channel audio signal comprising a first channel and a second channel, wherein the first channel and second channel are created using a beamformer and a microphone array, the first channel representing audio from a first direction, and the second channel representing audio from a second direction;
- creating a first sequence of feature vectors for the first channel and a second sequence of feature vectors for the second channel;
- performing speech recognition using the first sequence of feature vectors and the second sequence of feature vectors, wherein performing speech recognition comprises:
- generating a first hypothesis using a speech recognition model and a first feature vector of the first sequence of feature vectors;
- determining a first confidence score for the first hypothesis;
- generating a second hypothesis using the speech recognition model and a second feature vector of the second sequence of feature vectors, wherein the second hypothesis is subsequent to the first hypothesis in a speech recognition result network;
- determining a second confidence score for the second hypothesis;
- determining that the first confidence score is greater than the second confidence score;
- determining a speech recognition output using the first hypothesis;
- determining that the first direction is associated with a direction of a speaker;
- determining, at a later time, a new sequence of feature vectors corresponding to further audio from the first direction;
- performing speech recognition on the new sequence of feature vectors to obtain a new hypothesis;
- and
- increasing a confidence score of the new hypothesis.
11 more elements
A computing device, comprising:
- at least one processor;
- a memory device including instructions operable to be executed by the at least one processor to perform a set of actions, configuring the at least one processor:
- to receive a multiple-channel audio signal comprising a first channel and a second channel, the first channel and second channel created using a beamformer and a microphone array, the first channel representing audio from a first direction, and the second channel representing audio from a second direction;
- to create a first sequence of feature vectors for the first channel and a second sequence of feature vectors for the second channel;
- to process a first feature vector of the first sequence of feature vectors, the first feature vector corresponding to a first frame, to generate a first speech recognition hypothesis;
- to process a second feature vector of the second sequence of feature vectors to generate a second speech recognition hypothesis, the second feature vector corresponding to a second frame, wherein the first frame and second frame correspond to a first time;
- to determine a first confidence score for the first speech recognition hypothesis;
- to determine a second confidence score for the second speech recognition hypothesis;
- to select the first speech recognition hypothesis based on the first confidence score being higher than the second confidence score;
- to output speech recognition results using the first speech recognition hypothesis;
- to determine that the first direction is associated with a direction of a speaker;
- to determine, at a later time, a new sequence of feature vectors corresponding to further audio from the first direction;
- to perform speech recognition on the new sequence of feature vectors to obtain a new speech recognition hypothesis;
- and
- to increase a confidence score of the new speech recognition hypothesis.
12 more elements
A non-transitory computer-readable storage medium storing processor-executable instructions for controlling a computing device, comprising:
- program code to receive a multi-channel audio signal comprising a first channel and a second channel, the first channel comprising audio from a first direction and the second channel comprising audio from a second direction;
- program code to create a first sequence of acoustic features for the first channel and a second sequence of acoustic features for the second channel;
- program code to process acoustic features from the first sequence corresponding to a first time frame to generate a first speech recognition hypothesis;
- program code to process acoustic features from the second sequence corresponding to the first time frame to generate a second first speech recognition hypothesis;
- program code to determine a first confidence score for the first speech recognition hypothesis;
- program code to determine a second confidence score for the second speech recognition hypothesis;
- program code to select the first speech recognition hypothesis based on the first confidence score being higher than the second confidence score;
- program code to output speech recognition results using the first speech recognition hypothesis;
- program code to determine that the first direction is associated with a direction of a speaker;
- program code to determine, at a later time, a new sequence of feature vectors corresponding to further audio from the first direction;
- program code to perform speech recognition on the new se sequence of feature vectors to obtain a new speech recognition hypothesis;
- and
- program code to increase a confidence score of the new speech recognition hypothesis.
10 more elements
Dependencies are inferred from the claim text.