# US 9,883,311 B2 · Rendering of audio objects using discontinuous rendering-matrix updates > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,883,311 B2 - Document ID: US9883311B2 - Status: Granted - Kind: B2 - Application number: 14/898,737 - Filed: 2014-06-23 - Published: 2016-05-19 - Granted: 2018-01-30 - Earliest claimed: 2013-06-28 - Assignees: Dolby Laboratories Licensing Corporation - Applicants: Dolby Laboratories Licensing Corporation - Inventors: Dirk Jeroen Breebaart; David S. McGrath; Rhonda Wilson - Web page: https://patentgrep.com/patent/US9883311B2 ## CPC - H04S 3/02: of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other - G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing - G10L 19/032: Quantisation or dequantisation of spectral components - H04S 7/30: {Control circuits for electronic adaptation of the sound field} - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - H04S: STEREOPHONIC SYSTEMS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract An audio playback system generates output signals for multiple channels of acoustic transducers by applying a rendering matrix to data representing the aural content and spatial characteristics of audio objects, so that the resulting sound field creates accurate listener impressions of the spatial characteristics. Matrix coefficients are updated to render moving objects. Discontinuous updates of the rendering matrix coefficients are controlled according to psychoacoustic principles to reduce audible artifacts. The updates may also be managed to control the amount of data needed to perform the updates. ## Claims ### Claim 1 A method for processing audio information that includes object data, wherein the method comprises: receiving one or more signals that convey the object data representing aural content and spatial metadata for each of one or more audio objects, wherein the spatial metadata contains data representing a location in space relative to a reference position in a playback system; processing the object data and configuration information to calculate rendering matrix coefficients for use in rendering signals in the playback system, wherein the configuration information describes a configuration of acoustic transducers in a set of acoustic transducers for the playback system; calculating a measure of update performance from the calculated rendering matrix coefficients and the object data according to psychoacoustic principles relating to the audibility of artifacts created by changes to rendering matrix coefficients, and deriving matrix update parameters from the measure of update performance; generating updated matrix coefficient values in response to the rendering matrix coefficients and the matrix update parameters; updating the rendering matrix coefficients in response to the updated matrix coefficient values; and either assembling an encoded representation of the object data and the rendering matrix coefficients into an encoded output signal, or applying the rendering matrix to the object data representing the aural content of audio objects to generate audio output signals representing the aural content of rendered audio objects for respective audio channels. ### Claim 2 (depends on 1) The method of claim 1, wherein: the measure of update performance comprises a measure of perceived distortion that would result from updating the rendering matrix with the calculated rendering matrix coefficients; and the matrix update parameters are derived to reduce magnitudes of changes in rendering matrix coefficients in response to the measure of perceived distortion to reduce audibility of artifacts generated by the coefficient changes. ### Claim 3 (depends on 2) The method of claim 2 that comprises: receiving one or more signals that convey bed channel data representing aural content for each of one or more audio channels, wherein each audio channel corresponds to an unvarying position relative to the reference position; wherein: the measure of perceived distortion is calculated also from the bed channel data; and either an encoded representation of the bed channel data is assembled into the encoded output signal, or the applying of the rendering matrix also includes combining with bed channel data to generate audio output signals representing the combined aural content of bed channel data and rendered audio objects for respective audio channels. ### Claim 4 (depends on 2) The method of claim 2, wherein magnitudes of changes in rendering matrix coefficients are controlled by one or more update-limit parameters established in response to an estimated perceived distortion resulting from the changes in rendering matrix coefficients. ### Claim 5 (depends on 4) The method of claim 4, wherein the one or more update-limit parameters are set not to reduce magnitudes of changes in rendering matrix coefficients when a psychoacoustic model determines its associated audio object is inaudible. ### Claim 6 (depends on 1) The method of claim 1 that comprises deriving the matrix update parameters to reduce a rate at which changes in rendering matrix coefficients are performed, wherein the rate is controlled to reduce audibility of resulting artifacts generated by the coefficient changes. ### Claim 7 (depends on 6) The method of claim 6, wherein: the measure of update performance comprises an estimated change in perceived accuracy of spatial characteristics of audio objects rendered by the rendering matrix that would result from updating the rendering matrix with the calculated rendering matrix coefficients; and performing the changes in rendering matrix coefficients only if the change in perceived accuracy exceeds a threshold. ### Claim 8 (depends on 1) The method of claim 1, wherein each coefficient in the rendering matrix has an associated gain factor, and wherein the method comprises: adjusting each gain factor so that output of the rendering matrix does not exceed a maximum allowable level. ### Claim 9 (depends on 1) The method of claim 1 that comprises driving one or more acoustic transducers in the set of acoustic transducers in response to each audio output signal. ### Claim 10 An audio information processing apparatus for processing audio information that includes object data, wherein the audio information processing apparatus comprises one or more processors configured to: receive one or more signals that convey the object data representing aural content and spatial metadata for each of one or more audio objects, wherein the spatial metadata contains data representing a location in space relative to a reference position in a playback system; process the object data and configuration information to calculate rendering matrix coefficients for use in rendering signals in the playback system, wherein the configuration information describes a configuration of acoustic transducers in a set of acoustic transducers for the playback system; calculate a measure of update performance from the calculated rendering matrix coefficients and the object data according to psychoacoustic principles relating to the audibility of artifacts created by changes to rendering matrix coefficients, and deriving matrix update parameters from the measure of update performance; generate updated matrix coefficient values in response to the rendering matrix coefficients and the matrix update parameters; update the rendering matrix coefficients in response to the updated matrix coefficient values; and either assemble an encoded representation of the object data and the rendering matrix coefficients into an encoded output signal, or apply the rendering matrix to the object data representing the aural content of audio objects to generate audio output signals representing the aural content of rendered audio objects for respective audio channels. ### Claim 11 (depends on 10) The audio information processing apparatus of claim 10, wherein: the measure of update performance comprises a measure of perceived distortion that would result from updating the rendering matrix with the calculated rendering matrix coefficients; and the matrix update parameters are derived to reduce magnitudes of changes in rendering matrix coefficients in response to the measure of perceived distortion to reduce audibility of artifacts generated by the coefficient changes. ### Claim 12 (depends on 11) The audio information processing apparatus of claim 11, wherein the one or more processors are further configured to receive one or more signals that convey bed channel data representing aural content for each of one or more audio channels, wherein each audio channel corresponds to an unvarying position relative to the reference position; wherein: the measure of perceived distortion is calculated also from the bed channel data; and either an encoded representation of the bed channel data is assembled into the encoded output signal, or the applying of the rendering matrix also includes combining with bed channel data to generate audio output signals representing the combined aural content of bed channel data and rendered audio objects for respective audio channels. ### Claim 13 (depends on 11) The audio information processing apparatus of claim 11, wherein magnitudes of changes in rendering matrix coefficients are controlled by one or more update-limit parameters established in response to an estimated perceived distortion resulting from the changes in rendering matrix coefficients. ### Claim 14 (depends on 13) The audio information processing apparatus of claim 13, wherein the one or more update-limit parameters are set not to reduce magnitudes of changes in rendering matrix coefficients when a psychoacoustic model determines its associated audio object is inaudible. ### Claim 15 (depends on 10) The audio information processing apparatus of claim 10, wherein the audio information process is further configured to derive the matrix update parameters to reduce a rate at which changes in rendering matrix coefficients are performed, wherein the rate is controlled to reduce audibility of resulting artifacts generated by the coefficient changes. ### Claim 16 (depends on 15) The audio information processing apparatus of claim 15, wherein: the measure of update performance comprises an estimated change in perceived accuracy of spatial characteristics of audio objects rendered by the rendering matrix that would result from updating the rendering matrix with the calculated rendering matrix coefficients; and the one or more processors are configured to perform the changes in rendering matrix coefficients only if the change in perceived accuracy exceeds a threshold. ### Claim 17 (depends on 10) The audio information processing apparatus of claim 10, wherein each coefficient in the rendering matrix has an associated gain factor, and wherein the one or more processors are further configured to adjust each gain factor so that output of the rendering matrix does not exceed a maximum allowable level. ### Claim 18 (depends on 10) The audio information processing apparatus of claim 10, wherein the audio information processing apparatus is further configured to drive one or more acoustic transducers in the set of acoustic transducers in response to each audio output signal. ### Claim 19 A non-transitory medium recording a program of instructions that is executable by a device to perform a method for processing audio information that includes object data, wherein the method comprises: receiving one or more signals that convey the object data representing aural content and spatial metadata for each of one or more audio objects, wherein the spatial metadata contains data representing a location in space relative to a reference position in a playback system; processing the object data and configuration information to calculate rendering matrix coefficients for use in rendering signals in the playback system, wherein the configuration information describes a configuration of acoustic transducers in a set of acoustic transducers for the playback system; calculating a measure of update performance from the calculated rendering matrix coefficients and the object data according to psychoacoustic principles relating to the audibility of artifacts created by changes to rendering matrix coefficients, and deriving matrix update parameters from the measure of update performance; generating updated matrix coefficient values in response to the rendering matrix coefficients and the matrix update parameters; updating the rendering matrix coefficients in response to the updated matrix coefficient values; and either assembling an encoded representation of the object data and the rendering matrix coefficients into an encoded output signal, or applying the rendering matrix to the object data representing the aural content of audio objects to generate audio output signals representing the aural content of rendered audio objects for respective audio channels. ### Claim 20 (depends on 19) The medium of claim 19, wherein: the measure of update performance comprises a measure of perceived distortion that would result from updating the rendering matrix with the calculated rendering matrix coefficients; and the matrix update parameters are derived to reduce magnitudes of changes in rendering matrix coefficients in response to the measure of perceived distortion to reduce audibility of artifacts generated by the coefficient changes.