US 12,749,478 B2 · Enhanced speech-to-text performance with mixed languages

Abstract

Training a mixed language speech recognition model, and speech recognition performed by the model, can be enhanced. Model manager can control training and/or operation of the model. Mixed language data generation manager can generate an enhanced mixed language dataset that can enhance training of the model. Fine tuner can facilitate configuring hyperparameters of the model. Audio-based information and transcript that are representative of the dataset can be applied to the model to facilitate model training. FAL evaluator can determine fidelity, accuracy, and latency of performance of speech recognition on the audio-based information and transcript by the model. Based on such determination, mixed language data generation process and/or hyperparameters can be updated to enhance further training of the model to enhance fidelity, accuracy, and/or latency regarding performance of speech recognition by the model. Model manager can control one or more iterations of model training.

Claims20

Dependent claims

A method, comprising:

generating, by a trained model of a system comprising at least one processor, mixed language data based on instruction data relating to mixed language data generation and seed topic data relating to a group of seed topics, wherein the mixed language data comprises respective first textual words in a first language and respective second textual words in a second language that are determined based on respective simulated conversations between respective simulated speakers, and wherein the respective second textual words are interspersed with the respective first textual words;
and
as part of training of a speech recognition model, determining, by the trained model of the system, quality metrics indicative of a speech recognition performance of the speech recognition model in recognizing information relating to spoken mixed language words representative of the mixed language data, and generating textual transcript mixed language words representative of the spoken mixed language words, wherein the quality metrics are indicative of a fidelity, an accuracy, and a latency relating to the speech recognition performance.

A system, comprising:

at least one memory that stores computer executable components;
and
at least one processor that executes computer executable components stored in the at least one memory, wherein the computer executable components comprise:
a mixed language data generator that determines and generates mixed language data based on instruction information relating to mixed language data generation and seed topic information relating to a group of seed topics, wherein the mixed language data comprises respective first textual words in a first language and respective second textual words in a second language that are determined based on respective simulated conversations between respective simulated speakers, and wherein the respective second textual words are commingled with the respective first textual words;
and
an evaluator that determines, in connection with training of a speech recognition model, quality metric values indicative of a speech recognition performance of the speech recognition model in recognizing information relating to spoken mixed language words representative of the mixed language data, and generating textual transcript mixed language words representative of the spoken mixed language words, wherein the quality metric values relate to a fidelity, an accuracy, and a latency associated with the speech recognition performance of the speech recognition model.

3 more elements

A non-transitory machine-readable medium, comprising executable instructions that, when executed by at least one processor, facilitate performance of operations, comprising:

determining mixed language data based on instruction data relating to mixed language data generation and seed topic data relating to a group of seed topics, wherein the mixed language data comprises respective first textual words in a first language and respective second textual words in a second language that are determined based on respective emulated conversations between respective emulated speakers, and wherein the respective second textual words are intermingled with the respective first textual words within a same sentence;
and
in connection with training of a speech recognition model, determining quality metric rating values indicative of a speech recognition performance of the speech recognition model in recognizing information relating to spoken mixed language words representative of the mixed language data, and generating textual transcript mixed language words representative of the spoken mixed language words, wherein the quality metric rating values are representative of a fidelity, an accuracy, and a latency associated with the speech recognition performance of the speech recognition model.

Dependencies are inferred from the claim text.

Enhanced speech-to-text performance with mixed languages

Life of this patent

Enhanced speech-to-text performance with mixed languages

Life of this patent

Abstract

Claims20