# US 2025/0168583 A1 · Audio processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0168583 A1 - Document ID: US20250168583A1 - Status: Published application - Kind: A1 - Application number: 19/030,595 - Filed: 2025-01-17 - Published: 2025-05-22 - Earliest claimed: 2019-09-24 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Juha VILKAMO; Riitta VÄÄNÄNEN; Sampo VESA; Mikko-Ville LAITINEN; Jussi VIROLAINEN - Web page: https://patentgrep.com/patent/US20250168583A1 ## CPC - H04S 7/302: {Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence)} - H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} - H04S 2400/09: Electronic reduction of distortion of stereophonic sound systems - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] - H04S: STEREOPHONIC SYSTEMS - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract According to an example embodiment, a method for processing an input audio signal in accordance with spatial metadata so as to play back a spatial audio signal in a device in dependence of at least one sound reproduction characteristic of the device is provided, the method comprising: obtaining said input audio signal and said spatial metadata; obtaining said at least one sound reproduction characteristic of the device; rendering a first portion of the spatial audio signal using a first type playback procedure applied on the input audio signal in dependence of the spatial metadata, wherein the first portion comprises sound directions within a front region of the spatial audio signal; and rendering a second portion of the spatial audio signal using a second type playback procedure applied on the input audio signal in dependence of the spatial metadata and in dependence of said at least one sound reproduction characteristic, wherein the second portion comprises sound directions that are not included in the first portion and where the second type playback procedure is different from the first playback procedure and involves cross-talk cancellation processing. ## Claims ### Claim 1 A method for processing an input audio signal in accordance with spatial metadata so as to play back a spatial audio signal in a device in dependence of at least one sound reproduction characteristic of the device, the method comprising: obtaining, by the device, said input audio signal; obtaining, by the device, said spatial metadata, separate from obtaining said input audio signal, wherein said input audio signal and said spatial metadata convey a spatial audio image; obtaining said at least one sound reproduction characteristic of the device; deriving a first target covariance matrix based on said spatial metadata, wherein the first target covariance matrix represents sounds included in a first portion of the spatial audio image and comprises sound directions within a front region of the spatial audio image; deriving a second target covariance matrix based on said spatial metadata and the at least one sound reproduction characteristic of the device, wherein the second target covariance matrix represents sounds included in a second portion of the spatial audio image, wherein the second portion comprises sound directions that are not included in the first portion; and rendering, by the device, the spatial audio signal based on the input audio signal and in dependence of the first and second target covariance matrices. ### Claim 2 (depends on 1) A method according to claim 1, wherein rendering the spatial audio signal comprises: rendering a first portion of the spatial audio signal using a first type playback procedure applied on the input audio signal in dependence of the first target covariance matrix; and rendering a second portion of the spatial audio signal using a second type of playback procedure applied on the input audio signal in dependence of the second target covariance matrix, wherein the second type playback procedure is different than the first type playback procedure. ### Claim 3 (depends on 1) A method according to claim 1, wherein rendering the spatial audio signal comprises rendering the spatial audio signal using first and second type playback procedures applied on the input audio signal in dependence of the first and second target covariance matrices, respectively. ### Claim 4 (depends on 1) A method according to claim 1, wherein the processing is carried out separately in a plurality of frequency sub-bands. ### Claim 5 (depends on 1) A method according to claim 1, wherein said spatial metadata comprises, for one or more frequency sub-bands, a respective sound direction parameter, and a respective energy ratio parameter. ### Claim 6 (depends on 1) A method according to claim 1, wherein the second portion also represents non-directional sounds of the spatial audio image. ### Claim 7 (depends on 1) A method according to claim 1, wherein the at least one sound reproduction characteristic comprises respective definitions of loudspeaker positions in relation to a reference position with respect to the device, and wherein the method further comprises defining a range of sound directions that belong to the front region based on the loudspeaker positions. ### Claim 8 (depends on 3) A method according to claim 3, wherein the first type playback procedure comprises an amplitude panning procedure. ### Claim 9 (depends on 3) A method according to claim 3, wherein the second type playback procedure comprises cross-talk cancellation processing. ### Claim 10 (depends on 1) A method according to claim 1, comprising: deriving a covariance matrix and an energy measure based on the input audio signal; deriving a combined target covariance matrix based on the first and second target covariance matrices; deriving, based on the covariance matrix and on the combined target covariance matrix, a mixing matrix that, when applied to the input audio signal, results in a modified audio signal having a covariance matrix that is similar to the combined target covariance matrix; deriving the spatial audio signal based on the input audio signal and the mixing matrix. ### Claim 11 (depends on 10) A method according to claim 10, wherein deriving the first target covariance matrix comprises: deriving, based on a sound direction parameter included in the spatial metadata, an energy divisor value that indicates an extent of inclusion in the first portion of the spatial audio image; determining, based on the sound direction parameter, panning gains; and deriving the first target covariance matrix based on the energy measure, on the panning gains, on the energy divisor value and on an energy ratio parameter included in the spatial metadata. ### Claim 12 (depends on 10) A method according to claim 10, wherein deriving the second target covariance matrix comprises: deriving, based on a sound direction parameter included in the spatial metadata and on the at least one sound reproduction characteristic, an energy divisor value that indicates an extent of inclusion in the first portion of the spatial audio image; determining, based on a sound direction parameter included in the spatial metadata, a head-related transfer function (HRTF); deriving, based on HRTFs spanning across a predefined range of sound directions, a diffuse field covariance matrix; and deriving the second target covariance matrix based on the energy measure, on the HRTF, on the diffuse field covariance matrix and on an energy ratio parameter included in the spatial metadata. ### Claim 13 (depends on 1) A method according to claim 1, wherein deriving the first target covariance matrix comprises multiplying the first portion of the spatial audio image using a gain value that is based on predefined equalization information included in the at least one sound reproduction characteristic. ### Claim 14 (depends on 1) A method according to claim 1, wherein deriving the second target covariance matrix comprises: deriving a set of cross-talk cancelling gains based on reference transfer functions included in the at least one sound reproduction characteristic; and applying the set of cross-talk cancelling gains to the second portion of the spatial audio signal. ### Claim 15 (depends on 1) A method according to claim 1, further comprising: deriving a covariance matrix and an energy measure based on the input audio signal; deriving an extended first target covariance matrix based on the first target covariance matrix and using a gain value that is based on predefined equalization information included in the at least one sound reproduction characteristic; deriving an extended second target covariance matrix based on the second target covariance matrix and on cross-talk cancelling gains; deriving a target covariance matrix as a combination of the extended first target covariance matrix and the extended second target covariance matrix; deriving, based on the covariance matrix and on the target covariance matrix, a mixing matrix that, when applied to the input audio signal, results in a modified audio signal having a covariance matrix that is similar to the target covariance matrix; and deriving an output audio signal, for playback by the device, as a product of the input audio signal and the respective mixing matrix. ### Claim 16 An apparatus for processing an input audio signal in accordance with spatial metadata so as to play back a spatial audio signal in a device in dependence of at least one sound reproduction characteristic of the device, the apparatus comprising at least one processor and at least one memory including computer program code, when executed by the at least one processor, cause the apparatus to: obtain said input audio signal; obtain said spatial metadata, separate from obtaining said input audio signal, wherein said input audio signal and said spatial metadata convey a spatial audio image; obtain said at least one sound reproduction characteristic of the device; derive a first target covariance matrix based on said spatial metadata, wherein the first target covariance matrix represents sounds included in a first portion of the spatial audio image and comprises sound directions within a front region of the spatial audio image; derive a second target covariance matrix based on said spatial metadata and the at least one sound reproduction characteristic of the device, wherein the second target covariance matrix represents sounds included in a second portion of the spatial audio image, wherein the second portion comprises sound directions that are not included in the first portion; and render the spatial audio signal based on the input audio signal and in dependence of the first and second target covariance matrices. ### Claim 17 (depends on 16) An apparatus according to claim 16, wherein the apparatus is caused to render the spatial audio signal by: rendering a first portion of the spatial audio signal using a first type playback procedure applied on the input audio signal in dependence of the first target covariance matrix; and rendering a second portion of the spatial audio signal using a second type of playback procedure applied on the input audio signal in dependence of the second target covariance matrix, wherein the second type playback procedure is different than the first type playback procedure. ### Claim 18 (depends on 16) An apparatus according to claim 16, wherein the apparatus is caused to render the spatial audio signal by rendering the spatial audio signal using first and second type playback procedures applied on the input audio signal in dependence of the first and second target covariance matrices, respectively. ### Claim 19 (depends on 18) An apparatus according to claim 18, wherein the first type playback procedure comprises an amplitude panning procedure and the second type playback procedure comprises cross-talk cancellation processing. ### Claim 20 (depends on 16) An apparatus according to claim 16, wherein the apparatus is further caused to: derive a covariance matrix and an energy measure based on the input audio signal; derive an extended first target covariance matrix based on the first target covariance matrix and using a gain value that is based on predefined equalization information included in the at least one sound reproduction characteristic; derive an extended second target covariance matrix based on the second target covariance matrix and on cross-talk cancelling gains; derive a target covariance matrix as a combination of the extended first target covariance matrix and the extended second target covariance matrix; derive, based on the covariance matrix and on the target covariance matrix, a mixing matrix that, when applied to the input audio signal, results in a modified audio signal having a covariance matrix that is similar to the target covariance matrix; and derive an output audio signal, for playback by the device, as a product of the input audio signal and the respective mixing matrix.