Multi-dialect and multilingual speech recognition

A system comprising:

• one or more computers;
• and
• one or more computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the following operations:
• receiving, by the one or more computers of the automated speech recognition system, audio data indicating audio characteristics of an utterance;
• providing, by the one or more computers of the automated speech recognition system, input features determined based on the audio data to a speech recognition model that has been trained to output score indicating the likelihood of linguistic units for each of multiple different languages or dialects, wherein the speech recognition model has been trained using cluster adaptive training, with each of the multiple languages or dialects corresponding to a separate cluster, and wherein the speech recognition model is configured to receive different identifiers as input to the speech recognition model to specify the different clusters corresponding to the respective languages or dialects;
• receiving, by the one or more computers of the automated speech recognition system, output that the speech recognition model generated in response to receiving the input features determined based on the audio data;
• and
• providing, as an output of the automated speech recognition system, a transcription of the utterance generated based on the output of the speech recognition model,
• wherein the speech recognition model has been trained using multi-task learning using:
• a first objective function corresponding to grapheme prediction;
• and
• a second objective function corresponding to a language or dialect classification cost, the first objective function and second objective function being weighted such that the speech recognition model is trained to learn hidden representations that are effective for both language and dialect classification and grapheme prediction.

One or more non-transitory computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the following operations:

• receiving, by the one or more computers of the automated speech recognition system, audio data indicating audio characteristics of an utterance;
• providing, by the one or more computers of the automated speech recognition system, input features determined based on the audio data to a speech recognition model that has been trained to output score indicating the likelihood of linguistic units for each of multiple different languages or dialects, wherein the speech recognition model has been trained using cluster adaptive training, with each of the multiple languages or dialects corresponding to a separate cluster, and wherein the speech recognition model is configured to receive different identifiers as input to the speech recognition model to specify the different clusters corresponding to the respective languages or dialects;
• receiving, by the one or more computers of the automated speech recognition system, output that the speech recognition model generated in response to receiving the input features determined based on the audio data;
• and
• providing, as an output of the automated speech recognition system, a transcription of the utterance generated based on the output of the speech recognition model,
• wherein the speech recognition model has been trained using multi-task learning using:
• a first objective function corresponding to grapheme prediction;
• and
• a second objective function corresponding to a language or dialect classification cost, the first objective function and second objective function being weighted such that the speech recognition model is trained to learn hidden representations that are effective for both language and dialect classification and grapheme prediction.