Automatic speech recognition using multi-dimensional models
AssigneeGoogle Inc.
InventorsBo Li, Tara N. Sainath
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for automatic speech recognition using multi-dimensional models. In some implementations, audio data that describes an utterance is received. A transcription for the utterance is determined using an acoustic model that includes a neural network having first memory blocks for time information and second memory blocks for frequency information. The transcription for the utterance is provided as output of an automated speech recognizer.
Claims25
A method performed by one or more computers, the method comprising:
- receiving, by the one or more computers, audio data that describes an utterance;
- determining, by the one or more computers, a transcription for the utterance, comprising processing the audio data using a neural network that has been trained as an acoustic model, the neural network having first memory blocks for time information and second memory blocks for frequency information, the first memory blocks being different from the second memory blocks, wherein the first memory blocks are time-LSTM blocks that each have a state, and wherein the second memory blocks are frequency-LSTM blocks that each have a state and a corresponding frequency step in a sequence of multiple frequency steps, wherein the states are determined for each of a sequence of multiple time steps;
- wherein, for each of at least some of the frequency-LSTM blocks, the frequency-LSTM block determines its state using the state of the time-LSTM block corresponding to the same frequency step at the previous time step;
- and
- wherein, for each of at least some of the time-LSTM blocks, the time-LSTM block determines its state using the state of the frequency-LSTM block corresponding to the same time step and the previous frequency step;
- and
- providing, by the one or more computers, the transcription for the utterance as output of an automated speech recognizer.
(canceled)
(canceled)
(canceled)
A system comprising:
- one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
- receiving, by the one or more computers, audio data that describes an utterance;
- determining, by the one or more computers, a transcription for the utterance, comprising processing the audio data using a neural network that has been trained as an acoustic model, the neural network having first memory blocks for time information and second memory blocks for frequency information, the first memory blocks being different from the second memory blocks, wherein the first memory blocks are time-LSTM blocks that each have a state, and wherein the second memory blocks are frequency-LSTM blocks that each have a state and a corresponding frequency step in a sequence of multiple frequency steps, wherein the states are determined for each of a sequence of multiple time steps;
- wherein, for each of at least some of the frequency-LSTM blocks, the frequency-LSTM block determines its state using the state of the time-LSTM block corresponding to the same frequency step at the previous time step;
- and
- wherein, for each of at least some of the time-LSTM blocks, the time-LSTM block determines its state using the state of the frequency-LSTM block corresponding to the same time step and the previous frequency step;
- and
- providing, by the one or more computers, the transcription for the utterance as output of an automated speech recognizer.
(canceled)
(canceled)
One or more non-transitory computer-readable storage media storing a computer program, the program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
- receiving, by the one or more computers, audio data that describes an utterance;
- determining, by the one or more computers, a transcription for the utterance, comprising processing the audio data using a neural network that has been trained as an acoustic model, the neural network having first memory blocks for time information and second memory blocks for frequency information, the first memory blocks being different from the second memory blocks, wherein the first memory blocks are time-LSTM blocks that each have a state, and wherein the second memory blocks are frequency-LSTM blocks that each have a state and a corresponding frequency step in a sequence of multiple frequency steps, wherein the states are determined for each of a sequence of multiple time steps;
- wherein, for each of at least some of the frequency-LSTM blocks, the frequency-LSTM block determines its state using the state of the time-LSTM block corresponding to the same frequency step at the previous time step;
- and
- wherein, for each of at least some of the time-LSTM blocks, the time-LSTM block determines its state using the state of the frequency-LSTM block corresponding to the same time step and the previous frequency step;
- and
- providing, by the one or more computers, the transcription for the utterance as output of an automated speech recognizer.
Dependencies are inferred from the claim text.