# US 12,183,351 B2 · Audio encoding/decoding with transform parameters > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,351 B2 - Document ID: US12183351B2 - Status: Granted - Kind: B2 - Application number: 17/762,709 - Filed: 2020-09-22 - Published: 2022-11-17 - Granted: 2024-12-31 - Earliest claimed: 2019-09-23 - Assignees: Dolby Laboratories Licensing Corporation - Applicants: Dolby Laboratories Licensing Corporation - Inventors: Dirk Jeroen Breebaart; Alex Brandmeyer; Poppy Anne Carrie Crum; McGregor Steele Joyner; David Mcgrath; Andrea Fanelli; Rhonda J. Wilson - Web page: https://patentgrep.com/patent/US12183351B2 ## CPC - G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing - H04S 7/308: {Electronic adaptation dependent on speaker or headphone connection} - H04S 1/007: {in which the audio signals are in digital form (data reduction aspects thereof based on psychoacoustics G10L19/02)} - H04S 7/306: {For headphones} - H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04S: STEREOPHONIC SYSTEMS ## Abstract Encoding/decoding techniques where multiple transform parameter sets are encoded together with a rendered playback presentation of an input audio content. The multiple transform parameters are used on the decoder side to transform the playback presentation to provide a personalized binaural playback presentation optimized for an individual listener with respect to their hearing profile. This may be achieved by selection or combination of the data present in the metadata streams. ## Claims ### Claim 1 A method of encoding an input audio content having one or more audio components, wherein each audio component is associated with a spatial location, the method including the steps of: rendering said input audio content into an audio playback presentation, said audio playback presentation intended for reproduction on an audio reproduction system; determining a set of M binaural representations by applying M sets of transfer functions to the input audio content, wherein the M sets of transfer functions are based on a collection of individual binaural playback profiles; computing M sets of transform parameters enabling a transform from said audio playback presentation to M approximations of said M binaural representations, wherein said M sets of transform parameters are determined by minimizing a difference between said M binaural representations and said M approximations, wherein M>1; and encoding said audio playback presentation and said M sets of transform parameters for transmission to a decoder. ### Claim 2 (depends on 1) The method according to claim 1, wherein said M binaural representations are M individual binaural playback presentations intended for reproduction on headphones, said M individual binaural playback presentations corresponding to M individual playback profiles. ### Claim 3 (depends on 1) The method according to claim 1, wherein said M binaural representations are M canonical binaural playback presentations intended for reproduction on headphones, said M canonical binaural playback presentations representing a larger collection of individual playback profiles. ### Claim 4 (depends on 1) The method according to claim 1, wherein said M sets of transfer functions are M sets of head related transfer functions. ### Claim 5 (depends on 1) The method according to claim 1, wherein said audio playback presentation is a primary binaural playback presentation intended to be reproduced on headphones, and wherein said M binaural representations are M signal pairs each representing a difference between said primary binaural playback presentation and a binaural playback presentation corresponding to an individual playback profile. ### Claim 6 (depends on 5) The method according to claim 5, wherein said M signal pairs are rendered by M principal component analysis (PCA) basis functions. ### Claim 7 (depends on 1) The method according to claim 1, wherein said audio playback presentation is intended for a loudspeaker system, and wherein M binaural representations include a primary binaural presentation intended to be reproduced on headphones, and M−1 signal pairs each representing a difference between said primary binaural playback presentation and a binaural playback presentation corresponding to an individual playback profile. ### Claim 8 (depends on 1) The method according to claim 1, wherein the number M of transfer functions sets is different for different frequency bands. ### Claim 9 (depends on 1) The method according to claim 1, wherein the step of applying the personalized set of transform parameters to the audio playback presentation is performed by applying a linear gain N×2 matrix to the audio playback presentation, where N is the number of channels in the audio playback presentation, and the elements of the matrix are formed by the transform parameters. ### Claim 10 (depends on 1) A non-transitory computer-readable medium storing computer program code portions configured to perform the steps of claim 1 when executed on a processor. ### Claim 11 A method of decoding a personalized binaural playback presentation from an audio bitstream, the method including the steps of: receiving and decoding an audio playback presentation, said audio playback presentation intended for reproduction on an audio reproduction system; receiving and decoding M sets of transform parameters enabling a transform from said audio playback presentation to M approximations of M binaural representations, wherein said M sets of transform parameters have been determined by an encoder to minimize a difference between said M binaural representations and said M approximations generated by application of the transform parameters to the audio playback presentation, wherein M>1; combining said M sets of transform parameters into a personalized set of transform parameters; and applying the personalized set of transform parameters to the audio playback presentation, to generate said personalized binaural playback presentation. ### Claim 12 (depends on 11) The method according to claim 11, wherein the step of combining said M sets of transform parameters includes selecting a personalized set as one of the M sets. ### Claim 13 (depends on 11) The method according to claim 11, wherein the step of combining said M sets of transform parameters includes forming a personalized set as a linear combination of the M sets. ### Claim 14 (depends on 11) The method according to claim 11, wherein said audio playback presentation is a primary binaural playback presentation intended to be reproduced on headphones, and wherein said M sets of transform parameters enabling a transform from said audio playback presentation into M signal pairs each representing a difference between said primary binaural playback presentation and a binaural playback presentation corresponding to an individual playback profile, and wherein the step of applying the personalized set of transform parameters to the primary binaural playback presentation includes: forming a personalized binaural difference by applying the personalized set of transform parameters as a linear gain 2×2 matrix to the primary binaural playback presentation, and summing said personalized binaural difference and the primary binaural playback presentation. ### Claim 15 (depends on 11) The method according to claim 11, wherein said audio playback presentation is intended to be reproduced on loudspeakers, and wherein a first set of said M sets of transform parameters enables a transform from said audio playback presentation into an approximation of a primary binaural presentation, and remaining sets of transform parameters enable a transform from said audio playback presentation into M−1 signal pairs each representing a difference between said primary binaural playback presentation and a binaural playback presentation corresponding to an individual playback profile, and wherein the step of applying the personalized set of transform parameters to the primary binaural playback presentation includes: forming a primary binaural presentation by applying the first set of transform parameters to the audio playback presentation, forming a personalized binaural difference by applying the personalized set of transform parameters as a linear gain 2×2 matrix to said primary binaural playback presentation, and summing said personalized binaural difference and the primary binaural playback presentation. ### Claim 16 (depends on 15) The method according to claim 15, wherein the step of applying the first set of transform parameters to the audio playback presentation is performed by applying a linear gain N×2 matrix to the audio playback presentation, where N is the number of channels in the audio playback presentation and the elements of the matrix are formed by the transform parameters. ### Claim 17 (depends on 11) A non-transitory computer-readable medium storing computer program product including computer program code portions configured to perform the steps of claim 11 when executed on a processor. ### Claim 18 An encoder for encoding an input audio content having one or more audio components, wherein each audio component is associated with a spatial location, the encoder comprising: a first renderer for rendering said input audio content into an audio playback presentation, said audio playback presentation intended for reproduction on an audio reproduction system; a second renderer for determining a set of M binaural representations by applying M sets of transfer functions to the input audio content, wherein the M sets of transfer functions are based on a collection of individual binaural playback profiles; a parameter estimation module for computing M sets of transform parameters enabling a transform from said audio playback presentation to M approximations of said M binaural representations, wherein said M sets of transform parameters are determined by minimizing a difference between said M binaural representations and said M approximations, wherein M>1; and an encoding module for encoding said audio playback presentation and said M sets of transform parameters for transmission to a decoder. ### Claim 19 (depends on 18) The encoder according to claim 18, wherein said second renderer is configured to render M individual binaural playback presentations intended for reproduction on headphones, said M individual binaural playback presentations corresponding to M individual playback profiles. ### Claim 20 (depends on 18) The encoder according to claim 18, wherein said second renderer is configured to render M canonical binaural playback presentations intended for reproduction on headphones, said M canonical binaural playback presentations representing a larger collection of individual playback profiles. ### Claim 21 (depends on 18) The encoder according to claim 18, wherein said first renderer is configured to render a primary binaural playback presentation intended to be reproduced on headphones, and wherein said second renderer is configured to render M signal pairs each representing a difference between said primary binaural playback presentation and a binaural playback presentation corresponding to an individual playback profile. ### Claim 22 (depends on 18) The encoder according to claim 18, wherein said first renderer I configured to render an audio playback presentation intended for a loudspeaker system, and wherein said second renderer is configured to render a primary binaural presentation intended to be reproduced on headphones, and M−1 signal pairs each representing a difference between said primary binaural playback presentation and a binaural playback presentation corresponding to an individual playback profile. ### Claim 23 A decoder for decoding a personalized binaural playback presentation from an audio bitstream, the decoder comprising: a decoding module for receiving said audio bitstream and decoding an audio playback presentation intended for reproduction on an audio reproduction system and M sets of transform parameters enabling a transform from said audio playback presentation to M approximations of M binaural representations, wherein M>1, wherein said M sets of transform parameters have been determined by minimizing a difference between said M binaural representations and said M approximations generated by application of the transform parameters to the audio playback presentation; a processing module for combining said M sets of transform parameters into a personalized set of transform parameters; and a presentation transformation module for applying the personalized set of transform parameters to the audio playback presentation, to generate said personalized binaural playback presentation. ### Claim 24 (depends on 23) The decoder according to claim 23, wherein said processing module is configured to select one of the M sets as said personalized. ### Claim 25 (depends on 23) The decoder according to claim 23, wherein said processing module is configured to form a personalized set as a linear combination of the M sets. ### Claim 26 (depends on 23) The decoder according to claim 23, wherein said audio playback presentation is a primary binaural playback presentation intended to be reproduced on headphones, and wherein said M sets of transform parameters enabling a transform from said audio playback presentation into M signal pairs each representing a difference between said primary binaural playback presentation and a binaural playback presentation corresponding to an individual playback profile, and wherein said presentation transformation module is configured to: form a personalized binaural difference by applying the personalized set of transform parameters as a linear gain 2×2 matrix to the primary binaural playback presentation, and sum said personalized binaural difference and said primary binaural playback presentation. ### Claim 27 (depends on 23) The decoder according to claim 23, wherein said audio playback presentation is intended to be reproduced on loudspeakers, and wherein a first set of said M sets of transform parameters enables a transform from said audio playback presentation into an approximation of a primary binaural presentation, and remaining sets of transform parameters enable a transform from said audio playback presentation into M−1 signal pairs each representing a difference between said primary binaural playback presentation and a binaural playback presentation corresponding to an individual playback profile, and wherein said presentation transformation module is configured to: form a primary binaural presentation by applying the first set of transform parameters to the audio playback presentation, form a personalized binaural difference by applying the personalized set of transform parameters as a linear gain 2×2 matrix to said primary binaural playback presentation, and sum said personalized binaural difference and the primary binaural playback presentation.