Phoneme-based contextualization for cross-lingual speech recognition in end-to-end models

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• receiving audio data encoding an utterance spoken by a native speaker of a first language;
• receiving a biasing term list comprising one or more terms in a second language different than the first language;
• processing, using a speech recognition model, acoustic features derived from the audio data to generate speech recognition scores for both wordpieces and corresponding phoneme sequences in the first language;
• rescoring the speech recognition scores for the phoneme sequences based on the one or more terms in the biasing term list, and
• executing, using the speech recognition scores for the wordpieces and the rescored speech recognition scores for the phoneme sequences, a decoding graph to generate a transcription for the utterance.