Characterizing accuracy of ensemble models for automatic speech recognition by determining a predetermined number of multiple ASR engines based on their historical performance

A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to perform the steps of:

• generating features representing transcriptions produced by multiple automatic speech recognition (ASR) engines from voice activity in the recording and a best transcription of the recording produced by an ensemble model from the transcriptions;
• applying a machine learning model to the features to produce a score representing an accuracy of the best transcription, wherein applying the machine learning model to the features to produce the score representing the accuracy of the best transcription comprises:
• determining a predetermined number of the multiple ASR engines having historical performance in generating transcripts that are better than the historical performance of a remaining number of the multiple ASR engines under predetermined conditions as contributor ASR engines, the pre-determined conditions comprising a selected metadata associated with the recording,
• determining a remaining number of the multiple ASR engines as selector ASR engines, and
• applying the machine learning model to produce the score representing error rates between the best transcription and the transcriptions from the selector ASR engines;
• and
• storing the score in association with the best transcription.

A system, comprising:

• a memory that stores instructions;
• and
• a processor that is coupled to the memory and, when executing the instructions, is configured to:
• generate features representing transcriptions produced by multiple automatic speech recognition (ASR) engines from voice activity in the recording and a best transcription of the recording produced by an ensemble model from the transcriptions;
• apply a machine learning model to the features to produce a score representing an accuracy of the best transcription, wherein applying the machine learning model to the features to produce the score representing the accuracy of the best transcription comprises:
• determining a predetermined number of the multiple ASR engines having historical performance in generating transcripts that are better than the historical performance of a remaining number of the multiple ASR engines under predetermined conditions as contributor ASR engines, the pre-determined conditions comprising a selected metadata associated with the recording,
• determining a remaining number of the multiple ASR engines as selector ASR engines, and
• applying the machine learning model to produce the score representing error rates between the best transcription and the transcriptions from the selector ASR engines;
• and
• store the score in association with the best transcription.