Back to results
US 10,332,508 B1Granted
Confidence checking for speech processing and query answering
AssigneeAmazon Technologies, Inc.
InventorBjörn Hoffmeister
Life of this patent
Estimating the term…
Abstract
An automatic speech recognition (ASR) system uses recurrent neural network (RNN) encoding to create a feature vector corresponding to a word sequence ASR result where the feature vector incorporates data from different hierarchies (i.e., frame level, phone level, etc.) of the ASR processing. The feature vector may be used with a trained classifier to confirm that the ASR result was correct, or to otherwise assign a confidence score to the ASR results.
Claims20
A computer implemented method for confirming automatic speech recognition results, the method comprising:
- performing, on audio data comprising a plurality of audio frames, automatic speech recognition (ASR) to obtain ASR results, the ASR results comprising:
- a word sequence comprising a word, the word sequence representing a highest scoring word sequence corresponding to the audio data in the ASR results,
- a confidence associated with the word,
- a phone associated with the word,
- a confidence associated with the phone, and
- an indication that an audio frame of the plurality of audio frames is associated with the phone;
- determining audio frame data comprising at least the indication and mel-frequency cepstral coefficients corresponding to the audio frame;
- processing the audio frame data with a first recurrent neural network (RNN) encoder to determine a first feature vector corresponding to the audio frame;
- determining phone data comprising at least a duration of the phone, an identifier corresponding to the phone, and the confidence associated with the phone;
- processing the phone data and the first feature vector with a second RNN encoder to determine a second feature vector corresponding to the phone;
- determining word data representing comprising at least a duration of the word, a number of phones in the word, and the confidence associated with the word;
- processing the word data and the second feature vector with a third RNN encoder to determine a third feature vector corresponding to the word;
- determining word sequence data comprising at least a duration of the word sequence and a number of words in the word sequence;
- processing the word sequence data and the third feature vector with a fourth RNN encoder to determine a fourth feature vector corresponding to the word sequence;
- processing the fourth feature vector using a trained neural-network classifier to determine a score representing a confidence that the word sequence is an accurate textual representation of the audio data;
- and
- determining that the score is above a threshold.
14 more elements
A computer implemented method comprising:
- performing, on audio data, automatic speech recognition (ASR) to obtain ASR results, the ASR results comprising at least a word sequence;
- determining, using the ASR results, word sequence data representing characteristics of the word sequence;
- processing the word sequence data with a recurrent neural network (RNN) encoder to determine a feature vector corresponding to the word sequence;
- and
- processing the feature vector using a trained neural-network classifier to determine that the word sequence is an accurate textual representation of the audio data.
2 more elements
A computing system comprising:
- at least one processor;
- and
- a memory device including instructions operable to be executed by the at least one processor to configure the system to:
- perform, on audio data, automatic speech recognition (ASR) to obtain ASR results, the ASR results comprising at least a word sequence;
- determine, using the ASR results, word sequence data representing characteristics of the word sequence;
- process the word sequence data with a recurrent neural network (RNN) encoder to determine a feature vector corresponding to the word sequence;
- and
- process the feature vector using a trained neural-network classifier to determine that the word sequence is an accurate textual representation of the audio data.
5 more elements
Dependencies are inferred from the claim text.