# US 2019/0378533 A1 · Computing devices and methods for converting audio signals to text > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0378533 A1 - Document ID: US20190378533A1 - Status: Published application - Kind: A1 - Application number: 16/425,548 - Filed: 2019-05-29 - Published: 2019-12-12 - Earliest claimed: 2018-06-07 - Inventors: Gene Chao - Web page: https://patentgrep.com/patent/US20190378533A1 ## CPC - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 13/00: Speech synthesis; Text to speech systems - G10L 17/00: Speaker identification or verification techniques - G10L 17/02: Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction - G10L 21/10: Transforming into visible information - G10L 15/065: Adaptation - G10L 21/003: Changing voice quality, e.g. pitch or formants - G10L 25/15: the extracted parameters being formant information - G10L 2015/025: {Phonemes, fenemes or fenones being the recognition units} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Computer-implemented methods and computing devices for converting spoken language into text. Computer-implemented methods include receiving an input audio signal that includes spoken language uttered by a speaker, analyzing the input audio signal, and determining one or more differences between one or more measured formant values and one or more model formant values. The computer-implemented methods further may include identifying an optimal trained model for processing the input audio signal and/or transforming the input audio signal into a transformed audio signal that more closely matches the trained model. Computing devices for converting spoken language into text include a processing unit and a memory that stores non-transitory computer readable instructions that, when executed by the processing unit, cause the computing devices to perform the computer-implemented methods disclosed herein. ## Claims ### Claim 1 A computer-implemented method for improving the accuracy of voice to text conversion, the method comprising: receiving an input audio signal that includes spoken language uttered by a speaker; and analyzing the input audio signal; wherein the analyzing the input audio signal includes: comparing, with a computing device, one or more measured formant values to one or more model formant values, wherein each of the one or more measured formant values corresponds to a respective measured formant component of one or more measured formant components of an individual phoneme in the input audio signal, and wherein each of the one or more model formant values corresponds to a respective model formant component of one or more model formant components of the individual phoneme in a trained model of an automatic speech recognition (ASR) application; and determining, with the computing device, one or more differences between the one or more measured formant values and the one or more model formant values. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the one or more measured formant values correspond to the N lowest-frequency measured formant components of the individual phoneme of the input audio signal; wherein the one or more model formant values correspond to the N lowest-frequency model formant components of the individual phoneme in the trained model; and wherein N is an integer that is at least 2 and at most 4. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the one or more measured formant values further includes an Mth measured formant value corresponding to an Mth measured formant component of the individual phoneme in the input audio signal; wherein the one or more model formant values further includes an Mth model formant value corresponding to an Mth model formant component of the individual phoneme in the trained model; wherein the determining the one or more differences further includes determining one or more differences between the Mth measured formant value and the Mth model formant value; and wherein M is an integer that is at least 2 and at most 6. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the ASR application includes a plurality of trained models, and wherein the method further includes identifying an optimal trained model of the plurality of trained models for processing the input audio signal with the ASR application, wherein the identifying the optimal trained model includes identifying the trained model that represents speech characteristics that are most similar to speech characteristics of the input audio signal. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the analyzing the input audio signal includes repeating the determining the one or more differences between the one or more measured formant values and the one or more model formant values for each trained model of the plurality of trained models, and wherein the identifying the optimal trained model includes identifying which trained model of the plurality of trained models minimizes the one or more differences between the one or more measured formant values and the one or more model formant values. ### Claim 6 (depends on 4) The computer-implemented method of claim 4, further comprising generating transcription data with the ASR application, wherein the transcription data is based, at least in part, on the input audio signal, and wherein the generating the transcription data includes processing, with the ASR application, the input audio signal in accordance with the speech characteristics of the optimal trained model. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, the method further comprising: transforming the input audio signal into a transformed audio signal that more closely matches the trained model, wherein the transforming includes applying one or more transformations to the input audio signal, wherein the one or more transformations are based, at least in part, on the determining the one or more differences; and transmitting the transformed audio signal to the ASR application. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, further comprising generating transcription data with the ASR application, wherein the transcription data is based, at least in part, on the input audio signal, and wherein the generating the transcription data includes processing, with the ASR application, the transformed audio signal in accordance with the speech characteristics of the trained model. ### Claim 9 (depends on 7) The computer-implemented method of claim 7, wherein the transforming the input audio signal comprises modifying one or more frequency bands in the input audio signal, wherein each of the one or more frequency bands corresponds to a respective formant component of the individual phoneme. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, wherein the transforming the input audio signal comprises modifying one or more additional frequency bands in the input audio signal that correspond to respective formant components of an additional single phoneme that is different than the individual phoneme. ### Claim 11 (depends on 7) The computer-implemented method of claim 7, wherein the transforming the input audio signal is based, at least in part, on a transformation pattern that specifies relational transformation values for each of a plurality of individual phonemes. ### Claim 12 (depends on 11) The computer-implemented method of claim 11, wherein the transformation pattern specifies at least a first transformation to be applied to a first portion of the input audio signal and a second transformation to be applied to a second portion of the input audio signal, wherein the first portion of the input audio signal corresponds to at least one phoneme in the input audio signal, wherein the second portion of the input audio signal corresponds to at least an additional phoneme in the input audio signal, and wherein the at least one phoneme is different from the at least an additional phoneme. ### Claim 13 (depends on 7) The computer-implemented method of claim 7, further comprising, subsequent to the transforming the input audio signal, refining, with the computing device, one or both of the transformations and the transformed audio signal. ### Claim 14 (depends on 13) The computer-implemented method of claim 13, wherein the refining includes: measuring one or more transformed formant values corresponding to one or more transformed formant components in the transformed audio signal; comparing, with the computing device, the one or more transformed formant values to an additional one or more model formant values corresponding to an additional one or more model formant components of an additional phoneme in the trained model; determining, with the computing device, one or more transformed differences between the one or more transformed formant values and the additional one or more model formant values; and modulating the transformed audio signal to form a refined transformed audio signal that more closely matches speech characteristics of the trained model than does the transformed audio signal; wherein the modulating the transformed audio signal includes applying a refined transformation to the transformed audio signal; wherein the refined transformation is based, at least in part, on the one or more transformed differences. ### Claim 15 (depends on 13) The computer-implemented method of claim 13, further comprising generating transcription data with the ASR application, wherein the transcription data is based, at least in part, on the input audio signal; wherein the refining further includes: receiving, from the ASR application, the transcription data; generating, with the computing device, a synthesized audio signal that corresponds to the transcription data; comparing the synthesized audio signal to the transformed audio signal; and modulating the transformed audio signal based, at least in part, on the comparing the synthesized audio signal to the transformed audio signal. ### Claim 16 (depends on 1) The computer-implemented method of claim 1, further comprising: generating transcription data with the ASR application, wherein the transcription data is based, at least in part, on the input audio signal, and wherein the generating the transcription data includes determining and transcribing spoken language included in the input audio signal into a textual transcription of the spoken language; and generating, with the computing device, closed captioning data that are to be shown in association with portions of a media signal, wherein the generating the closed captioning data is based, at least in part, on the generating the transcription data; wherein the closed captioning data include the textual transcription. ### Claim 17 (depends on 16) The computer-implemented method of claim 16, wherein the analyzing the input audio signal further comprises detecting an additional speaker in the input audio signal, wherein the detecting the additional speaker includes detecting a change in the speech characteristics of the input audio signal, and wherein the closed captioning data include a textual indication that the speaker has changed from the speaker to the additional speaker. ### Claim 18 (depends on 17) The computer-implemented method of claim 17, wherein the detecting the additional speaker includes detecting the identity of the additional speaker, and wherein the closed captioning data include a textual identification of the identity of the additional speaker. ### Claim 19 (depends on 1) A computing device, comprising: a processor; and a memory that stores non-transitory computer readable instructions that, when executed by the processor, cause the computing device to perform the method of claim 1. ### Claim 20 (depends on 1) A non-transitory computer readable medium storing instructions that, when executed by a processor, cause a computing device to perform the computer-implemented method of claim 1.