# US 12,118,472 B2 · Methods and systems for training and providing a machine learning model for audio compensation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,472 B2 - Document ID: US12118472B2 - Status: Granted - Kind: B2 - Application number: 18/521,987 - Filed: 2023-11-28 - Published: 2024-05-30 - Granted: 2024-10-15 - Earliest claimed: 2022-11-28 - Assignees: Treble Technologies - Applicants: Treble Technologies - Inventors: Finnur Pind; Jesper Pedersen - Web page: https://patentgrep.com/patent/US12118472B2 ## CPC - G06F 30/23: using finite element methods [FEM] or finite difference methods [FDM] - G06N 3/094: Adversarial learning - G06F 30/25: using particle-based methods - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - H04S 7/30: {Control circuits for electronic adaptation of the sound field} - H04S 7/302: {Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence)} - H04S 7/303: {Tracking of listener position or orientation} - H04S 7/305: {Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence)} - H04S 7/307: {Frequency adjustment, e.g. tone control (H04S7/301 takes precedence)} - G06F 2111/10: Numerical modelling - G06F 2119/10: Noise analysis or noise optimisation - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - H04S 2400/15: Aspects of sound capture and related signal processing for recording or reproduction - H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] - H04S 2420/11: Application of ambisonics in stereophonic audio systems - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04S: STEREOPHONIC SYSTEMS ## Abstract A computer implemented method for training a machine learning model for audio compensation includes: receiving 3D models of a plurality of rooms, each of the 3D models comprising at least one sound source and at least one acoustic property; receiving a plurality of impulse responses at a listening position in each of the plurality of rooms, training the machine learning model for audio compensation using at least the plurality of impulse responses as input. Methods and systems using the disclosed computer implemented method and its features are also disclosed. ## Claims ### Claim 1 A computer implemented method for training a machine learning model for audio compensation, the method comprising: receiving 3D models of a plurality of rooms, each of the 3D models comprising at least one position of at least one directive sound source configured to emit sound in a defined direction in each of the 3D models of the plurality of rooms, and at least one acoustic property of at least one boundary; receiving at least one listening position in each of the 3D models of the plurality of rooms; receiving a plurality of impulse responses at the at least one listening position in each of the 3D models of the plurality of rooms, wherein the plurality of impulse responses is obtained with a wave-based solver, wherein the wave-based solver determines the plurality of impulse responses of a wave-based propagation of an impulse emitted at the at least one directive sound source in the 3D models of the plurality of rooms, and received at the at least one listening position; and training the machine learning model using at least the plurality of impulse responses as input, to perform one or more of blind room acoustic modeling, dereverberation, echo-cancellation, speech recognition, beamforming, and virtual sound rendering. ### Claim 2 (depends on 1) The computer implemented method according to claim 1, further comprising pre-processing the plurality of impulse responses to generate a plurality of modified impulse responses for training the machine learning model, wherein the modified impulse responses replace the plurality of impulse responses in the step of receiving the plurality of impulse responses. ### Claim 3 (depends on 2) The computer implemented method according to claim 2, wherein the pre-processing comprises applying a filter to enhance a voice range in the impulse response. ### Claim 4 (depends on 3) The computer implemented method according to claim 3, wherein the voice range is one of between 3 KHz-17 KHz and between 350 Hz-17 KHz. ### Claim 5 (depends on 1) The computer implemented method according to claim 1, further comprising providing a plurality of reverberating audio signals and using the plurality of reverberating audio signals as input for training the machine learning model. ### Claim 6 (depends on 5) The computer implemented method according to claim 5, wherein the reverberating audio signal is provided by convolving each of the plurality of impulse responses with a base audio signal. ### Claim 7 (depends on 5) The computer implemented method according to claim 5, wherein the reverberating audio signal is provided by recording an audio signal received at an at least one speaker of an audio device. ### Claim 8 (depends on 1) The computer implemented method according to claim 1, further comprising using the 3D models of the plurality of rooms as input for training the machine learning model. ### Claim 9 (depends on 7) The computer implemented method according to claim 7, further comprising receiving a digital model of the audio device, and using the digital model of the audio device as an input for training the machine learning model. ### Claim 10 (depends on 1) The computer implemented method according to claim 1, further comprising using the listening position in each of the plurality of rooms as input for training the machine learning model. ### Claim 11 (depends on 5) The computer implemented method according to claim 5, wherein the training comprises reestablishing a base audio signal as an output of the machine learning model. ### Claim 12 (depends on 11) The computer implemented method according to claim 11, further comprising using the 3D models of the plurality of rooms and the listening position in each of the plurality of rooms as input for training the machine learning model, and reestablishing the base audio signal at the listening position as an output when training the machine learning model. ### Claim 13 (depends on 5) The computer implemented method according to claim 5, wherein training the machine learning model comprises generating a compensation impulse response as an output. ### Claim 14 (depends on 13) The computer implemented method according to claim 13, wherein generating the compensation impulse response is based on the reverberation audio signals and a base audio signal as an input for training the machine learning model. ### Claim 15 (depends on 13) The computer implemented method according to claim 13, wherein the 3D models of the plurality of rooms and the listening position in each of the plurality of rooms are used as input for training the machine learning model, the method further comprising generating the compensation impulse response at the listening point as an output. ### Claim 16 (depends on 7) The computer implemented model according to claim 7, further comprising receiving a digital model of the audio device. ### Claim 17 (depends on 1) The computer implemented method according to claim 1, wherein the machine learning model comprises a neural network. ### Claim 18 (depends on 17) The computer implemented method according to claim 17, wherein the neural network comprises an auto encoder for encoding any of the inputs to the model or for generating a compressed input for use in training the machine learning model for audio compensation. ### Claim 19 (depends on 17) The computer implemented method according to claim 17, wherein the neural network further comprises a general adversarial network (GAN) for generating any one of the inputs for training the machine learning model for audio compensation. ### Claim 20 (depends on 17) The computer implemented method according to claim 17 wherein the neural network comprises at least one of a deep neural network, a convolutional neural networks and a transformer for training the machine learning model for audio compensation. ### Claim 21 (depends on 1) A method for providing a machine learning model for audio compensation in an audio device, having at least one microphone, wherein the machine learning model is trained by the method according to claim 1, the method comprising: receiving an audio signal at the at least one microphone and generating a compensated audio signal using the machine learning model for audio compensation. ### Claim 22 (depends on 1) A method for providing a machine learning based audio compensation in an audio device having at least one microphone, wherein the machine learning based audio compensation has been trained using a computer implemented method for training a machine learning model according to claim 1, the method comprising: receiving an audio signal at the at least one microphone and generating a compensated audio signal using the trained machine learning model on the received audio signal. ### Claim 23 (depends on 21) The method according to claim 21, wherein the machine learning model has further been trained to generate a compensation impulse response as an output and wherein generating the compensated audio signal includes convolving the received audio signal with the compensation impulse response. ### Claim 24 (depends on 22) An audio device comprising at least one microphone and a processing system for applying a machine learning based audio compensation according to the method of claim 22, wherein the audio device is configured to receive an audio signal at the at least one microphone and to apply the machine learning based audio compensation to the audio signal to generate a compensated audio signal. ### Claim 25 (depends on 24) The audio device according to claim 24, wherein the audio device further comprises a communication module configured to transmit the compensated audio signal. ### Claim 26 (depends on 25) The audio device according to claim 25, wherein the audio device is further configured to transmit the compensated audio signal to at least one of a cloud storage, an internet storage, and a network storage. ### Claim 27 (depends on 25) The audio device according to claim 25, wherein the audio device is further configured to transmit the compensated audio signal to a remote audio device, wherein the remote audio device comprises at least one remote speaker for outputting the compensated audio signal. ### Claim 28 A system for training a machine learning model for audio compensation, the system comprising: a computer system having a processor coupled to a memory, the processor configured to: receive 3D models of a plurality of rooms, each of the 3D models including at least one position of at least one directive sound source configured to emit sound in a defined direction in each of the 3D models of the plurality of rooms, and at least one acoustic property of at least one boundary; receive at least one listening position in each of the 3D models of the plurality of rooms; receive a plurality of impulse responses at the at least one listening position in each of the 3D models of the plurality of rooms, wherein the plurality of impulse responses are obtained with a wave-based solver, wherein the wave-based solver determines the plurality of impulse responses of a wave-based propagation of an impulse emitted at the at least one directive sound source in the 3D models of the plurality of rooms, and received at the at least one listening position; and training the machine learning model for audio compensation using at least the plurality of impulse responses as input, wherein the audio compensation compensates for reverberations or echo.