Tied and reduced RNN-T

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware and storing instructions that when executed on the data processing hardware causes the data processing hardware to perform operations comprising:
• receiving a sequence of acoustic frames characterizing an utterance;
• at each of a plurality of time steps subsequent to an initial time step:
• receiving, as input to a prediction network of a recurrent neural network-transducer (RNN-T) model, a sequence of non-blank symbols output by a final Softmax layer;
• generating, by a single head of the prediction network, a sequence of embeddings for the sequence of non-blank symbols received as input at the corresponding time step, each corresponding embedding in the sequence of embeddings corresponds to a respective non-blank symbol in the sequence of non-blank symbols;
• weighting, by the single head of the prediction network, each corresponding embedding in the sequence of embeddings based on a respective position embedding assigned to the respective non-blank symbol that corresponds to the corresponding embedding;
• generating, as output from the single head of the prediction network, a weighted average of the sequence of weighted embeddings;
• generating, by a projection layer of the prediction network, a projection output for the weighted average of the sequence of weighted embeddings output from the single head of the prediction network;
• and
• normalizing the projection output for the weighted average of the sequence of weighted embeddings to provide, as output from the prediction network, a single embedding vector at the corresponding time step;
• and
• generating, by the final Softmax layer, as output, a speech recognition result for the sequence of acoustic frames based the single embedding vectors output from the prediction network at each of the plurality of time steps.