# US 10,872,602 B2 · Training of acoustic models for far-field vocalization processing systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,602 B2 - Document ID: US10872602B2 - Status: Granted - Kind: B2 - Application number: 16/402,043 - Filed: 2019-05-02 - Published: 2019-11-28 - Granted: 2020-12-22 - Earliest claimed: 2018-05-24 - Assignees: Dolby Laboratories Licensing Corporation - Applicants: Dolby Laboratories Licensing Corporation - Inventors: Hadis Nosrati; David S. McGrath; Richard J. Cartwright - Web page: https://patentgrep.com/patent/US10872602B2 ## CPC - G10L 15/06: Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice (G10L15/14 takes precedence) - G10L 15/187: Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams - G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G10L 21/003: Changing voice quality, e.g. pitch or formants - G10L 21/0232: Processing in the frequency domain - H04R 3/04: for correcting frequency response - G10L 15/20: Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) - G10L 2015/088: {Word spotting} - G10L 2021/02082: {the noise being echo, reverberation of the speech} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract Computer-implemented methods for training an acoustic model for a far-field utterance processing system are provided. The acoustic model may be configured to map an input audio signal into linguistic or paralinguistic units. The training may involve imparting far-field acoustic characteristics upon near-field training vectors that include a plurality of near-microphone utterance signals. Imparting the far-field acoustic characteristics may involve generating a plurality of simulated room impulse responses, convolving one or more of the simulated room impulse responses with the near-field training vectors, to produce a plurality of simulated far-field utterance signals and saving the results of the training in one or more non-transitory memory devices corresponding with the acoustic model. Generating simulated room impulse responses may involve simulating room reverberation times but not simulating early reflections from room surfaces. ## Claims ### Claim 1 A computer-implemented audio processing method, comprising: training, by a control system that comprises one or more processors and one or more non-transitory memory devices, an acoustic model for a far-field vocalization processing system, the acoustic model configured to map an input audio signal into linguistic or paralinguistic units, wherein the training involves imparting far-field acoustic characteristics upon near-field training vectors, wherein the near-field training vectors comprise a plurality of near-microphone vocalization signals, and wherein imparting the far-field acoustic characteristics involves: generating, by the control system, a plurality of simulated room models, wherein the generating involves simulating diffuse room reverberation but not simulating individual early reflections from room surfaces; and imparting, via the control system, one or more of the simulated room models on the near-field training vectors, to produce a plurality of simulated far-field vocalization signals; and saving the plurality of simulated far-field vocalization signals of the training in one or more non-transitory memory devices corresponding with the acoustic model. ### Claim 2 (depends on 1) The audio processing method of claim 1, wherein the room models comprise room impulse responses and wherein the imparting involves time-domain convolution of the impulse responses with the near-field training vectors. ### Claim 3 (depends on 1) The audio processing method of claim 1, wherein the room models comprise room impulse responses and wherein the imparting involves frequency-domain convolution of the impulse responses with the near-field training vectors. ### Claim 4 (depends on 1) The audio processing method of claim 1, wherein imparting the far-field acoustic characteristics involves: receiving, by the control system, near-microphone vocalization signals; applying, by the control system, a filter bank to the near-microphone vocalization signals to produce a plurality of features, each feature corresponding to a different frequency band of the near-microphone vocalization signals; modifying, by the control system, original frequency band energies of each feature to produce modified features that represent energy contours of far-field vocalization signals across frequency bands; and providing the modified features to an acoustic modeling estimation process that is implemented by the control system, the acoustic modeling estimation process being configured to generate phoneme sequences, or sequences of other linguistic or paralinguistic units, corresponding to the near-field vocalization signals. ### Claim 5 (depends on 4) The audio processing method of claim 4, wherein the modifying involves: generating delayed and attenuated replicates of original frequency band energies of each feature; and applying one or more addition operators to the delayed and attenuated replicates. ### Claim 6 (depends on 5) The audio processing method of claim 5, wherein generating the attenuated replicates involves determining a gain based on one or more factors selected from a list of factors consisting of a reverberation time, a distance between a source and a microphone, a length of a time frame and a room size. ### Claim 7 (depends on 4) The audio processing method of claim 4, wherein the modifying involves applying one or more delay operators to original frequency band energies of each feature. ### Claim 8 (depends on 1) The audio processing method of claim 1, wherein generating the simulated room models involves simulating room impulse responses corresponding to a diffuse sound that decays over time according to a reverberation time. ### Claim 9 (depends on 1) The audio processing method of claim 1, wherein the far-field vocalization processing system comprises one or more of a far-field speech recognition system, a far-field speaker recognition system, a wakeword detection system, and an emotion recognition system. ### Claim 10 (depends on 1) The audio processing method of claim 1, wherein training the acoustic model comprises: extracting, via a speech analysis process implemented by the control system, features from the simulated far-field vocalization signals; and providing extracted features to an acoustic modeling estimation process configured to generate phoneme sequences, or sequences of other linguistic or paralinguistic units, corresponding to the simulated far-field vocalization signals. ### Claim 11 (depends on 10) The audio processing method of claim 10, wherein the control system implements the acoustic modeling estimation process according to one or more of: a Hidden Markov model, a Gaussian mixture model or a neural network. ### Claim 12 (depends on 1) The audio processing method of claim 1, wherein training the acoustic model comprises changing a physical state of at least one non-transitory memory device location corresponding with a near-field training vector. ### Claim 13 (depends on 1) The audio processing method of claim 1, wherein the far-field vocalization processing system comprises a far-field speech recognition system, further comprising implementing the far-field speech recognition system by: implementing, via the control system, the acoustic model; implementing, via the control system, a pronunciation model by mapping linguistic units produced by the acoustic model into words according to a pronunciation dictionary; and implementing, via the control system, a language model by estimating a probability of a word given a previous word sequence produced by the pronunciation model. ### Claim 14 (depends on 13) The audio processing method of claim 13, wherein implementing the far-field speech recognition system further comprises: receiving, by the control system, far-field speech; and producing, via the control system, a transcript corresponding to the far-field speech. ### Claim 15 (depends on 1) The audio processing method of claim 1, wherein the far-field vocalization processing system comprises one or more of a far-field talker recognition system, a wakeword detection system or an emotion recognition system, further comprising implementing the far-field vocalization processing system by implementing, via the control system, the acoustic model. ### Claim 16 (depends on 1) A non-transitory storage medium having software stored thereon, the software including instructions for controlling one or more processors to perform the audio processing method of claim 1. ### Claim 17 (depends on 1) An apparatus, comprising: an interface system; and a control system that comprises one or more processors and one or more non-transitory memory devices, the control system being configured to perform the audio processing method of claim 1. ### Claim 18 (depends on 1) An apparatus, comprising: an interface system; and a control system comprising one or more processors and one or more non-transitory storage media operatively coupled to the one or more processors, the control system configured to implement an acoustic model trained according to the method of claim 1, wherein the control system is further configured for implementing a far-field speech recognition system by: receiving, via the interface system, an audio signal corresponding to far-field speech; mapping, via the acoustic model, the audio signal into phonemes; mapping, via a pronunciation model and a language model, the phonemes into words; and outputting a representation of the words.