# US 2015/0146873 A1 · Rendering and Playback of Spatial Audio Using Channel-Based Audio Systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0146873 A1
- Document ID: US20150146873A1
- Status: Published application
- Kind: A1
- Application number: 14/409,440
- Filed: 2013-06-17
- Published: 2015-05-28
- Earliest claimed: 2012-06-19
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Christophe Chabanne; Brett Crockett; Spencer Hooks; Alan Seefeldt; Nicolas R. Tsingos; Mark Tuffy; Rhonda Wilson
- Web page: https://patentgrep.com/patent/US20150146873A1

## CPC

- H04S 7/305: {Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence)}
- H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- H04S 2400/03: Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 (H04S2400/01 takes precedence)
- H04S 2420/03: Application of parametric coding in stereophonic audio systems
- H04S: STEREOPHONIC SYSTEMS

## Abstract

Embodiments are described for a method and system of rendering and playing back spatial audio content using a channel-based format. Spatial audio content that is played back through legacy channel-based equipment is transformed into the appropriate channel-based format resulting in the loss of certain positional information within the audio objects and positional metadata comprising the spatial audio content. To retain this information for use in spatial audio equipment even after the audio content is rendered as channel-based audio, certain metadata generated by the spatial audio processor is incorporated into the channel-based data. The channel-based audio can then be sent to a channel-based audio decoder or a spatial audio decoder. The spatial audio decoder processes the metadata to recover at least some positional information that was lost during the down-mix operation by upmixing the channel-based audio content back to the spatial audio content for optimal playback in a spatial audio environment.

## Claims

### Claim at position 1

1-22. (canceled)

### Claim 23

A method of recovering spatial audio information rendered into a channel-based format for playback in a spatial audio environment, the channel-based format comprising a surround-sound format which includes a plurality of height speakers, the spatial audio environment comprising a system which includes the plurality of height speakers and a plurality of additional height speakers, the method comprising:
deriving metadata defining positional information of audio elements in a spatial audio processor that generates both channel-based and object-based information of the audio elements, wherein the metadata includes a matrix suitable for up-mixing a first set of channels using the plurality of height speakers to a second set of channels using the plurality of height speakers and the plurality of additional height speakers for playback in the spatial audio environment, and wherein the matrix is also suitable for down-mixing the first set of channels to a third set of channels using no height speakers;
incorporating the metadata in a channel-based format; and
combining the metadata and channel-based information in a spatial audio decoder to facilitate playback of the audio elements in the spatial audio environment.

### Claim 24 (depends on 23)

The method of claim 23 wherein the channel-based format comprises a 7.1 surround-sound format.

### Claim 25 (depends on 23)

The method of claim 23 wherein the up-mixing matrix comprises a time-varying matrix of size M×2, and wherein the matrix is incorporated into the channel-based format with data specifying the number M corresponding to a total number of speakers in the spatial audio environment, and an assumed position of the M channels within the spatial audio environment.

### Claim 26 (depends on 25)

The method of claim 25 wherein the audio elements comprise audio objects that are transmitted to respective speakers whose positions correspond to those specified in the metadata.

### Claim 27 (depends on 23)

The method of claim 23 wherein the up-mixing matrix is selected to minimize a defined cost function that is defined relative to a plurality of reference signals.

### Claim 28 (depends on 23)

The method of claim 23 wherein the metadata supplements a first metadata set that includes metadata elements associated with an object-based stream of the spatial audio information, the metadata elements for each object-based stream specifying spatial parameters controlling the playback of a corresponding object-based sound, and comprising one or more of:
sound position, sound width, and sound velocity; and further wherein the first metadata set includes metadata elements associated with a channel-based stream of the spatial audio information, and
wherein the metadata elements associated with each channel-based stream comprises designations of surround-sound channels of the speakers in a speaker array in accordance with a defined surround-sound configuration.

### Claim 29 (depends on 28)

The method of claim 28 wherein the first metadata set includes metadata to enable upmixing or downmixing of at least one of the channel-based audio streams and the object-based audio streams in accordance with a change from a first configuration of the speaker array to a second configuration of the speaker array.

### Claim 30 (depends on 29)

The method of claim 29 wherein the speakers of the speaker array are placed at specific positions within the playback environment, and wherein metadata elements associated with each respective object-based stream specify that one or more sound components are rendered to a speaker feed for playback through a speaker nearest an intended playback location of the sound component, as indicated by the position metadata.

### Claim 31 (depends on 23)

The method of claim 23 further comprising computing a plurality of height channel signals as a weighted sum of a corresponding plurality of audio objects defined by the spatial audio information.

### Claim 32 (depends on 31)

The method of claim 31 wherein the height channels are static.

### Claim 33 (depends on 31)

The method of claim 31 wherein the height channels are dynamic and the audio objects have a time-varying trajectory in a height plane.

### Claim 34 (depends on 33)

The method of claim 33 further comprising deriving mixing coefficients corresponding to right and left front speaker heights, respectively as a function of trajectories relative to assumed speaker positions of two channels in the height plane.

### Claim 35 (depends on 34)

The method of claim 34 further comprising deriving a weighted sum of the object trajectories, wherein the weights are a function of the mixing coefficients along with a loudness measure of each audio object.

### Claim 36 (depends on 35)

The method of claim 35 further comprising defining the metadata elements using the mixing coefficients and weighted sum of the object trajectories.

### Claim 37 (depends on 23)

The method of claim 23 further comprising identifying an inflection point along a front height axis to define a panning point at which sound is switched to or from front height speakers to rear surround speakers.

### Claim 38 (depends on 37)

The method of claim 37 wherein the inflection point serves to define a point in which any sound element located between the front height speakers and the inflection point will be collapsed, and any sound element located between the inflection point and the rear height speakers will be stretched.

### Claim 39 (depends on 38)

The method of claim 38 wherein the metadata comprises elements defining a position of the inflection point.

### Claim 40 (depends on 39)

The method of claim 39 wherein the position of the inflection point is expressed as coordinates of an enclosure defined within the spatial audio environment.

### Claim 41

An apparatus for recovering spatial audio information rendered into a channel-based format for playback in a spatial audio environment, the channel-based format comprising a surround-sound format which includes a plurality of height speakers, the spatial audio environment comprising a system which includes the plurality of height speakers and a plurality of additional height speakers, the apparatus comprising a processor configured to:
derive metadata defining positional information of audio elements in a spatial audio processor that generates both channel-based and object-based information of the audio elements, wherein the metadata includes a matrix suitable for up-mixing a first set of channels using the plurality of height speakers to a second set of channels using the plurality of height speakers and the plurality of additional height speakers for playback in the spatial audio environment, and wherein the matrix is also suitable for down-mixing the first set of channels to a third set of channels using no height speakers;
incorporate the metadata in a channel-based format; and
combine the metadata and channel-based information in a spatial audio decoder to facilitate playback of the audio elements in the spatial audio environment.

### Claim 42

A non-transitory storage medium recording a program of instructions that is executable by a device for performing a method of recovering spatial audio information rendered into a channel-based format for playback in a spatial audio environment, the channel-based format comprising a surround-sound format which includes a plurality of height speakers, the spatial audio environment comprising a system which includes the plurality of height speakers and a plurality of additional height speakers, the method comprising:
deriving metadata defining positional information of audio elements in a spatial audio processor that generates both channel-based and object-based information of the audio elements, wherein the metadata includes a matrix suitable for up-mixing a first set of channels using the plurality of height speakers to a second set of channels using the plurality of height speakers and the plurality of additional height speakers for playback in the spatial audio environment, and wherein the matrix is also suitable for down-mixing the first set of channels to a third set of channels using no height speakers;
incorporating the metadata in a channel-based format; and
combining the metadata and channel-based information in a spatial audio decoder to facilitate playback of the audio elements in the spatial audio environment.
