Language Model-Based Synthetic Data Generation For Code-Switching Automatic Speech Recognition

A system comprising:

• one or more computer systems having one or more hardware processors;
• and
• instructions which, when executed by the one or more computer systems, cause the one or more computer systems to perform a set of operations comprising:
• inputting, to a language model, one or more examples of code-switching between a first language and a second language;
• obtaining, from the language model, a plurality of code-switched N-grams, wherein an N-gram comprises a word from the first language and a word from the second language, wherein the plurality of code-switched N-grams are generated by the language model based at least on the one or more examples;
• generating, using the language model, a training dataset for a transformer-based ASR model, the training dataset comprising a plurality of code-switched training sentences that incorporate the plurality of code-switched N-grams;
• and
• fine-tuning a decoder portion of the transformer-based ASR model using the training dataset comprising the plurality of code-switched training sentences.

One or more non-transitory computer-readable media storing instructions which, when executed by one or more computer systems, cause the one or more computer systems to perform a set of operations comprising:

• inputting, to a language model, one or more examples of code-switching between a first language and a second language;
• obtaining, from the language model, a plurality of code-switched N-grams, wherein an N-gram comprises a word from the first language and a word from the second language, wherein the plurality of code-switched N-grams are generated by the language model based at least on the one or more examples;
• generating, using the language model, a training dataset for a transformer-based ASR model, the training dataset comprising a plurality of code-switched training sentences that incorporate the plurality of code-switched N-grams;
• and
• fine-tuning a decoder portion of the transformer-based ASR model using the training dataset comprising the plurality of code-switched training sentences.