Domain adaptation of automatic speech recognition systems using retrieval augmented generation

Abstract

Approaches presented herein provide for the generation of text transcripts of speech represented in audio data. In particular, an automatic speech recognition (ASR) model can be used together with a retrieval augmented generation (RAG) pipeline to provide for improvement of transcripts that include terminology related, or specific, to a specific knowledge domain. A knowledge base for a given domain can include a number of files or documents in a number of different formats (e.g., documents, images, and webpages) that do not need to be cleaned, classified, or curated. When an ASR generates a transcript where at least one word has a confidence level that falls below a confidence threshold, that transcript can be passed to a language model of the RAG pipeline which can use the retrieved domain-specific data to attempt to identify the appropriate words or terms to use to replace the words tagged as having low confidence.

Dependencies are inferred from the claim text.