Speech recognition with attention-based recurrent neural networks

A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

• obtaining an input acoustic sequence, the input acoustic sequence representing an utterance, and the input acoustic sequence comprising a respective acoustic feature representation at each of a first number of time steps;
• processing the input acoustic sequence using a first neural network to convert the input acoustic sequence into an alternative representation for the input acoustic sequence, wherein the first neural network is a pyramid Bidirectional Long Short Term Memory (BLSTM) Recurrent Neural Network (RNN), and wherein processing the input acoustic sequence using the first neural network to convert the input acoustic sequence into an alternative representation for the input acoustic sequence comprises:
• processing the input acoustic sequence through a bottom BLSTM layer to generate a BLSTM layer output;
• processing the BLSTM layer output through each a plurality of pyramid BLSTM layers, wherein consecutive outputs of each pyramid BLSTM layer are concatenated before being provided to the next pyramid BLSTM layer;
• processing the alternative representation for the input acoustic sequence using an attention-based RNN to generate, for each position in an output sequence order, a set of substring scores that includes a respective substring score for each substring in a set of substrings;
• and
• generating a sequence of substrings that represent a transcription of the utterance.

A computer-readable storage medium comprising instructions stored thereon that are executable by a processing device and upon such execution cause the processing device to perform operations comprising:

• obtaining an input acoustic sequence, the input acoustic sequence representing an utterance, and the input acoustic sequence comprising a respective acoustic feature representation at each of a first number of time steps;
• processing the input acoustic sequence using a first neural network to convert the input acoustic sequence into an alternative representation for the input acoustic sequence, wherein the first neural network is a pyramid Bidirectional Long Short Term Memory (BLSTM) Recurrent Neural Network (RNN), and wherein processing the input acoustic sequence using the first neural network to convert the input acoustic sequence into an alternative representation for the input acoustic sequence comprises:
• processing the input acoustic sequence through a bottom BLSTM layer to generate a BLSTM layer output;
• processing the BLSTM layer output through each a plurality of pyramid BLSTM layers, wherein consecutive outputs of each pyramid BLSTM layer are concatenated before being provided to the next pyramid BLSTM layer;
• processing the alternative representation for the input acoustic sequence using an attention-based RNN to generate, for each position in an output sequence order, a set of substring scores that includes a respective substring score for each substring in a set of substrings;
• and
• generating a sequence of substrings that represent a transcription of the utterance.