Systems and methods for processing bi-mode dual-channel sound data for automatic speech recognition models
AssigneeOptum, Inc.
InventorsJames J. Mou, Jun Li, Julie Zhu
Life of this patent
Estimating the term…
Abstract
Various embodiments of the present disclosure provide methods, apparatus, systems, computing devices, computing entities, and/or the like for pre-processing dual-channel voice data for an automatic speech recognition mode. The method comprises creating one or more spectrograms for each channel of the dual-channel voice data by applying fast Fourier transform and generating power spectral density. The one or more balanced power spectrograms are created by merging the spectrograms of the channels, and are provided as input for acoustic and language processing by an automatic speech recognition machine learning model.
Claims20
A computer-implemented method, in a data processing system comprising one or more processors and a memory, for automatic speech recognition, the computer-implemented method comprising:
- receiving, using one or more processors, dual-channel voice data;
- for each channel of the dual-channel voice data, extracting, using the one or more processors, one or more data sequences in frequency domain from the dual-channel voice data over one or more sliding time frame windows by applying fast Fourier transformation to the dual-channel voice data;
- generating, using the one or more processors, one or more spectrograms for each channel, a spectrogram comprising a power spectral density (PSD) waveform obtained by performing one or more forward linear prediction estimates on the one or more data sequences;
- determining, using the one or more processors, peak power of each of the one or more sliding time frame windows for each channel;
- determining, using the one or more processors, a most frequent power (MFP) based at least in part on the peak power of the one or more sliding time frame windows for each channel;
- generating, using the one or more processors, one or more balanced input spectrograms based at least in part on the one or more spectrograms of each channel and the most frequent power for each channel;
- generating, using the one or more processors and an acoustic machine learning model, character probabilities based at least in part on the one or more balanced input spectrograms;
- generating, using the one or more processors and a language machine learning model, a transcription output based at least in part on the character probabilities;
- generating, using the one or more processors and a natural language processing machine learning model, and based at least in part on the transcription output, a natural language processing output for the dual-channel voice data;
- and
- performing, using the one or more processors, one or more prediction-based actions based at least in part on the natural language processing output.
An apparatus for performing speech recognition processing, the apparatus comprising at least one processor and at least one memory including program code, the at least one memory and the program code configured to, with the processor, cause the apparatus to at least:
- receive dual-channel voice data;
- for each channel of the dual-channel voice data, extract one or more data sequences in frequency domain from the dual-channel voice data over one or more sliding time frame windows by applying fast Fourier transformation to the dual-channel voice data;
- generate one or more spectrograms for each channel, a spectrogram comprising a power spectral density (PSD) waveform obtained by performing one or more forward linear prediction estimates on the one or more data sequences;
- determine peak power of each of the one or more sliding time frame windows for each channel;
- determine a most frequent power (MFP) based at least in part on the peak power of the one or more sliding time frame windows for each channel;
- generate one or more balanced input spectrograms based at least in part on the one or more spectrograms of each channel and the most frequent power for each channel;
- generate, using an acoustic machine learning model, character probabilities based at least in part on the one or more balanced input spectrograms;
- generate, using a language machine learning model, a transcription output based at least in part on the character probabilities;
- generate, using a natural language processing machine learning model and based at least in part on the transcription output, a natural language processing output for the dual-channel voice data;
- and
- perform, using the one or more processors, one or more prediction-based actions based at least in part on the natural language processing output.
A computer program product for performing speech recognition processing, the computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions configured to:
- receive dual-channel voice data;
- for each channel of the dual-channel voice data, extract one or more data sequences in frequency domain from the dual-channel voice data over one or more sliding time frame windows by applying fast Fourier transformation to the dual-channel voice data;
- generate one or more spectrograms for each channel, a spectrogram comprising a power spectral density (PSD) waveform obtained by performing one or more forward linear prediction estimates on the one or more data sequences;
- determine peak power of each of the one or more sliding time frame windows for each channel;
- determine a most frequent power (MFP) based at least in part on the peak power of the one or more sliding time frame windows for each channel;
- generate one or more balanced input spectrograms based at least in part on the one or more spectrograms of each channel and the most frequent power for each channel;
- generate, using an acoustic machine learning model, character probabilities based at least in part on the one or more balanced input spectrograms;
- generate, using a language machine learning model, a transcription output based at least in part on the character probabilities;
- generate, using a natural language processing machine learning model and based at least in part on the transcription output, a natural language processing output for the dual-channel voice data;
- and
- perform, using the one or more processors, one or more prediction-based actions based at least in part on the natural language processing output.
Dependencies are inferred from the claim text.