Systems and methods for employing alternate spellings for improving the recognition of rare words

A method of speech recognition using alternate spellings, the method comprising:

• receiving, by an automatic speech recognition with rare words (ASIRW) module, a custom vocabulary comprising one or more user-submitted words or phrases;
• building, by the ASIRW module, identifying features for the custom vocabulary and identifying features for alternate spellings of the custom vocabulary;
• recognizing speech in an audio signal using the identifying features of the custom vocabulary and the identifying features of the alternate spellings;
• and
• determining a greatest likelihood mapping of the recognized speech to corresponding transcribed text.

A method of generating training data for an alternative spellings model, the method comprising:

• providing a text database comprising a ground-truth text and a plurality of sets of automatic speech recognition (ASR)-generated text for a same audio signal, the plurality of sets of ASR-generated text being produced by multiple ASR results or multiple ASR engines;
• aligning the ground-truth text and the plurality of sets of ASR-generated text using a text alignment tool to generate error pairs;
• filtering the error pairs based on uncommon words to produce filtered error pairs representing alternative words not ordinarily recognized via ASR;
• and providing the filtered error pairs to a translation model training process along with previously existing training configuration files to produce a trained translation model.

A method of improving recognition of customer-specified words in an automatic speech recognition (ASR) system, the method comprising:

• providing an ASR model that predicts pieces of text in units of one to eight characters in length, the ASR model configured to operate using one or more of connectionist temporal classification (CTC) decoding, CTC decoding guided by a weighted finite-state transducer (WFST), and WFST decoding alone;
• receiving a list of customer-specified words;
• generating new mappings from word pieces to the customer-specified words and to predicted alternate spellings of the customer-specified words;
• adding the new mappings to a WFST that includes language modeling information comprising mappings from word pieces to full words;
• and decoding an audio signal with the ASR model using the WFST with the new mappings to produce a transcription.

A system for training an alternative spellings model for a speech recognition system, the system comprising one or more processors configured to:

• identify, from automatic speech recognition (ASR) training data, a subset of utterances each containing at least one word appearing fewer than n times in the ASR training data;
• run an ASR model using connectionist temporal classification (CTC) decoding on audio segments corresponding to the subset of utterances to produce ASR output text;
• align the ASR output text with corresponding human-transcribed reference text using a text alignment tool to identify groups of contiguous errors comprising insertions, deletions, and substitutions;
• extract, from the groups of contiguous errors, error pairs comprising substitution errors and remove error pairs that include any of a set of most common words on a reference side to produce filtered error pairs;
• train a transformer-based encoder-decoder translation model on the filtered error pairs using the reference text tokenized into characters as input and the ASR output text tokenized with a tokenizer of the ASR model as output, to produce a trained translation model;
• and
• generate a transcript of an audio signal using the trained translation model.

A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• identifying, from automatic speech recognition (ASR) training data, a subset of utterances each containing at least one word appearing fewer than n times in the ASR training data;
• running an ASR model using connectionist temporal classification (CTC) decoding on audio segments corresponding to the subset of utterances to produce ASR output text;
• aligning the ASR output text with corresponding human-transcribed reference text using a text alignment tool to identify groups of contiguous errors comprising insertions, deletions, and substitutions;
• extracting error pairs comprising substitution errors and removing error pairs that include any of a set of most common words on a reference side to produce filtered error pairs;
• training a transformer-based encoder-decoder translation model on the filtered error pairs using the reference text tokenized into characters as input and the ASR output text tokenized with a tokenizer of the ASR model as output to produce a trained translation model;
• and
• generating a transcript of an audio signal using the trained translation model.