# US 12,192,741 B2 · Efficient HRTF approximation via multi-layer optimization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,741 B2 - Document ID: US12192741B2 - Status: Granted - Kind: B2 - Application number: 18/375,777 - Filed: 2023-10-02 - Published: 2024-01-25 - Granted: 2025-01-07 - Earliest claimed: 2022-03-18 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Mick Kekoa Marchan; Andrew Stewart Allen - Web page: https://patentgrep.com/patent/US12192741B2 ## CPC - H04S 1/002: {Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution (control circuits for electronic adaptation of the sound field H04S7/30)} - H04S 7/304: {For headphones} - G06F 17/142: {Fast Fourier transforms, e.g. using a Cooley-Tukey type algorithm} - G06N 20/00: Machine learning - H04R 5/033: Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)} - H04S 1/007: {in which the audio signals are in digital form (data reduction aspects thereof based on psychoacoustics G10L19/02)} - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - H04S 2400/13: Aspects of volume control, not necessarily automatic, in stereophonic sound systems - H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] - H04S: STEREOPHONIC SYSTEMS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract Techniques for using a network to decompose an HRTF data set to generate approximation data and to render an audio signal using the approximation data are disclosed herein. An input HRTF data set is fed into the network, which then generates approximation data that includes mixing channel gains, FIR filter coefficients, and basis filter shapes. This approximation data controls various components in the network. When the input HRTF data set is fed as input into the network, then the output of the network is an output approximated HRTF data set. The network iteratively fine tunes the approximation data until the output approximated HRTF data set sufficiently matches the input HRTF data set. After the approximation data is sufficiently tuned, the approximation data is later used by the network to render an audio signal. ## Claims ### Claim 1 A method for approximating head related transfer function (HRTF) filters when rendering an audio signal, said method comprising: providing an input HRTF data set as input into a network, which comprises basis filters, wherein the input HRTF data set includes a corresponding set of HRTF filters for each location in a set of locations; for each respective location in the set of locations, causing the network to use the input HRTF data set to iteratively learn approximation data comprising the following: (i) corresponding mixing channel gains that are used to control input to the mixing channels, (ii) coefficients that are used to control the FIR filters, and (iii) shapes of the basis filters; and storing a correlation between each location in the set of locations and specific values selected from the approximation data, wherein, by subsequently selecting a particular location in the set of locations, a corresponding set of values from the approximation data are also selected, said corresponding set of values include specific mixing channel gains, the coefficients, and the basis filter shapes. ### Claim 2 (depends on 1) The method of claim 1, wherein said method further includes rendering an input audio signal that is fed into the network by: identifying a specific location in the set of locations, where the specific location corresponds to the input audio signal; based on the specific location, selecting a particular set of mixing channel gains from the approximation data, selecting the coefficients from the approximation data, and selecting the basis filter shapes from the approximation data; configuring the network based on the selected particular set of mixing channel gains, based on the coefficients, and based on the basis filter shapes; after configuring the network, feeding the input audio signal as input into the network; causing the network to generate a left output signal and a right output signal based on the input audio signal; and playing the left output signal and the right output signal over a pair of speakers. ### Claim 3 (depends on 1) The method of claim 1, wherein each basis filter in said basis filters includes a single Fast Fourier Transform (FFT). ### Claim 4 (depends on 1) The method of claim 1, wherein: the network includes a per-source section comprising the mixing channels, the network further includes a fixed-compute section comprising the FIR filters and the FFTs, and the FFTs are included in only the fixed-compute section. ### Claim 5 (depends on 1) The method of claim 1, wherein the network includes 8 mixing channels per ear. ### Claim 6 (depends on 1) The method of claim 1, wherein the network is a 2-5-4 network that includes 2 mixing channels, 5 coefficients per FIR filter, and 4 basis filters. ### Claim 7 (depends on 1) The method of claim 1, wherein outputs of the basis filters are linearly combined in a frequency domain. ### Claim 8 (depends on 1) The method of claim 1, wherein the iterative learning includes iteratively modifying the approximation data until an error metric is less than an error threshold. ### Claim 9 (depends on 1) The method of claim 1, wherein the network includes a per-source section comprising the mixing channels. ### Claim 10 A method for approximating head related transfer function (HRTF) filters when rendering an audio signal, said method comprising: providing an input HRTF data set as input into a network, which comprises basis filters, wherein the input HRTF data set includes a corresponding set of HRTF filters for each location in a set of locations; for each respective location in the set of locations, causing the network to use the input HRTF data set to iteratively learn approximation data comprising the following: (i) corresponding mixing channel gains that are used to control input to the mixing channels, (ii) coefficients that are used to control the FIR filters, and (iii) shapes of the basis filters; storing a correlation between each location in the set of locations and specific values selected from the approximation data, wherein, by subsequently selecting a particular location in the set of locations, a corresponding set of values from the approximation data are also selected, said corresponding set of values include specific mixing channel gains, the coefficients, and the basis filter shapes; and rendering an input audio signal that is fed into the network. ### Claim 11 (depends on 10) The method of claim 10, wherein rendering the input audio signal includes: identifying a specific location in the set of locations, where the specific location corresponds to the input audio signal; based on the specific location, selecting a particular set of mixing channel gains from the approximation data, selecting the coefficients from the approximation data, and selecting the basis filter shapes from the approximation data; configuring the network based on the selected particular set of mixing channel gains, based on the coefficients, and based on the basis filter shapes; after configuring the network, feeding the input audio signal as input into the network; causing the network to generate a left output signal and a right output signal based on the input audio signal; and playing the left output signal and the right output signal over a pair of speakers. ### Claim 12 (depends on 10) The method of claim 10, wherein the network is a 2-5-4 network that includes 2 mixing channels, 5 coefficients per FIR filter, and 4 basis filters. ### Claim 13 (depends on 10) The method of claim 10, wherein outputs of the basis filters are linearly combined in a frequency domain. ### Claim 14 (depends on 10) The method of claim 10, wherein the iterative learning includes iteratively modifying the approximation data until an error metric is less than an error threshold. ### Claim 15 (depends on 10) The method of claim 10, wherein the network includes a per-source section comprising the mixing channels. ### Claim 16 A computer system that approximates head related transfer function (HRTF) filters when rendering an audio signal, said computer system comprising: one or more processors; and one or more hardware storage devices that store instructions that are executable by the one or more processors to cause the computer system to: provide an input HRTF data set as input into a network, which comprises basis filters, wherein the input HRTF data set includes a corresponding set of HRTF filters for each location in a set of locations; for each respective location in the set of locations, cause the network to use the input HRTF data set to iteratively learn approximation data comprising the following: (i) corresponding mixing channel gains that are used to control input to the mixing channels, (ii) coefficients that are used to control the FIR filters, and (iii) shapes of the basis filters; store a correlation between each location in the set of locations and specific values selected from the approximation data, wherein, by subsequently selecting a particular location in the set of locations, a corresponding set of values from the approximation data are also selected, said corresponding set of values include specific mixing channel gains, the coefficients, and the basis filter shapes; and render an input audio signal that is fed into the network. ### Claim 17 (depends on 16) The computer system of claim 16, wherein the network is further used to render a particular audio signal. ### Claim 18 (depends on 16) The computer system of claim 16, wherein each basis filter in said basis filters includes a single Fast Fourier Transform (FFT). ### Claim 19 (depends on 16) The computer system of claim 16, wherein the network includes 8 mixing channels per ear. ### Claim 20 (depends on 16) The computer system of claim 16, wherein outputs of the basis filters are linearly combined in a frequency domain.