Training Speech Recognition Models Using On-The-Fly Code-Switch Simulation With Monolingual Audio Data

One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause one or more computer systems to perform asset of operations comprising:

• receiving a first plurality of audio samples in a first language and a second plurality of audio samples in a second language;
• executing a training process for a speech recognition model, the training process comprising:
• selecting a first subset, of the first plurality of audio samples in the first language, and a second subset, of the second plurality of audio samples in the second language, for generating a plurality of synthetic code-switched audio samples;
• generating the plurality of synthetic code-switched audio samples based at least on combining the first subset and the second subset, wherein each synthetic code-switched audio sample, of the plurality of synthetic code-switched audio samples, includes audio content in both the first language and the second language;
• determining that a first subset of the plurality of synthetic code-switched audio samples satisfies one or more training constraints;
• determining that a second subset of the plurality of synthetic code-switched audio samples does not satisfy one or more training constraints;
• and
• training the speech recognition model based at least on the first subset of the plurality of synthetic code-switched audio samples.

A system comprising:

• one or more computer systems having one or more hardware processors;
• and
• instructions which, when executed by the one or more computer systems, cause the one or more computer systems to perform a set of operations comprising:
• receiving a first plurality of audio samples in a first language and a second plurality of audio samples in a second language;
• executing a training process for a speech recognition model, the training process comprising:
• selecting a first subset, of the first plurality of audio samples in the first language, and a second subset, of the second plurality of audio samples in the second language, for generating a plurality of synthetic code-switched audio samples;
• generating the plurality of synthetic code-switched audio samples based at least on combining the first subset and the second subset, wherein each synthetic code-switched audio sample, of the plurality of synthetic code-switched audio samples, includes audio content in both the first language and the second language;
• determining that a first subset of the plurality of synthetic code-switched audio samples satisfies one or more training constraints;
• determining that a second subset of the plurality of synthetic code-switched audio samples does not satisfy one or more training constraints;
• and
• training the speech recognition model based at least on the first subset of the plurality of synthetic code-switched audio samples.