# US 2017/0289730 A1 · Adaptive audio rendering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0289730 A1
- Document ID: US20170289730A1
- Status: Published application
- Kind: A1
- Application number: 15/199,664
- Filed: 2016-06-30
- Published: 2017-10-05
- Earliest claimed: 2016-03-30
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Robert Norman Heitkamp; Philip Andrew Edry; Ziyad Ibrahim; Paul J. Radek; Steven Wilssens
- Web page: https://patentgrep.com/patent/US20170289730A1

## CPC

- G10L 19/20: using sound class specific coding, hybrid encoders or object based coding
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- G06F 3/162: {Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs}
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
- H04L 65/1104: Session initiation protocol [SIP]
- H04S 3/002: {Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution (control circuits for electronic adaptation of the sound field H04S7/30)}
- H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- H04S 7/308: {Electronic adaptation dependent on speaker or headphone connection}
- H04S 7/303: {Tracking of listener position or orientation}
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04S: STEREOPHONIC SYSTEMS

## Abstract

The techniques disclosed herein can enable a system to coordinate the processing of object-based audio and channel-based audio generated by multiple applications. The system determines a spatialization technology to utilize based on contextual data. In some configurations, the contextual data can indicate the capabilities of one or more computing resources. In some configurations, the contextual data can also indicate preferences. The preferences, for example, can indicate user preferences for a type of spatialization technology, e.g., Dolby Atmos, over another type of spatialization technology, e.g., DTSX. Based on the contextual data, the system can select a spatialization technology and a corresponding encoder to process the input signals to generate a spatially encoded stream that appropriately renders the audio of multiple applications to an available output device. The techniques disclosed herein also allow a system to dynamically change the spatialization technologies during use.

## Claims

### Claim 1

A computing device, comprising:
a processor;
a computer-readable storage medium in communication with the processor, the computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the processor to:
receive contextual data indicating capabilities of an encoder or one or more endpoint devices;
select a spatialization technology based, at least in part, on the contextual data indicating capabilities of an encoder or one or more endpoint devices;
cause the encoder to generate a rendered output signal based on an input signal comprising object-based audio and channel-based audio processed by the selected spatialization technology; and
cause a communication of the rendered output signal from the encoder to the one or more endpoint devices.

### Claim 2 (depends on 1)

The computing device of claim 1, wherein the contextual data comprises one or more preferences, and wherein the selection of the spatialization technology is further based on the one or more preferences.

### Claim 3 (depends on 1)

The computing device of claim 1, wherein the contextual data comprises one or more preferences prioritizing a plurality of spatialization technologies, including a first spatialization technology as a first priority and a second spatialization technology as a second priority, and wherein selecting the spatialization technology comprises:
determining when the encoder and the one or more endpoint devices is compatible with the first spatialization technology;
determining the first spatialization technology as the selected spatialization technology when the encoder and the one or more endpoint devices is compatible with the first spatialization technology;
determining when the encoder and the one or more endpoint devices is compatible with the second spatialization technology;
determining the second spatialization technology as the selected spatialization technology when the encoder and the one or more endpoint devices is compatible with the second spatialization technology, and when the encoder or the one or more endpoint devices is not compatible with the first spatialization technology.

### Claim 4 (depends on 1)

The computing device of claim 1, wherein the contextual data comprises one or more preferences prioritizing a plurality of endpoint devices, including a first endpoint device as a first priority and a second endpoint device as a second priority, and wherein selecting the spatialization technology comprises:
determining that the first endpoint device of the one or more endpoint devices is compatible with the first spatialization technology;
determining when the first endpoint device is in communication with the encoder;
determining the first spatialization technology as the selected spatialization technology when it is determined that the first endpoint device is in communication with the encoder;
determining that the second endpoint device of the one or more endpoint devices is compatible with the second spatialization technology;
determining when the second endpoint device is in communication with the encoder; and
determining the second spatialization technology as the selected spatialization technology when it is determined that the second endpoint device is in communication with the encoder and when the first endpoint device is not in communication with the encoder.

### Claim 5 (depends on 1)

The computing device of claim 1, wherein selecting the spatialization technology comprises:
determining, based at least in part by the contextual data, that a first endpoint device of the one or more endpoint devices is compatible with a first spatialization technology;
determining when the first endpoint device is in communication with the encoder;
determining the first spatialization technology as the selected spatialization technology when it is determined that the first endpoint device is in communication with the encoder;
determining, based at least in part by the contextual data, that a second endpoint device of the one or more endpoint devices is compatible with a second spatialization technology;
determining when the second endpoint device is in communication with the encoder; and
determining the second spatialization technology as the selected spatialization technology when it is determined that the second endpoint device is in communication with the encoder.

### Claim 6 (depends on 1)

The computing device of claim 1, wherein the contextual data is generated, at least in part, by an application configured to receive an input, wherein the selection of the spatialization technology is further based on the input.

### Claim 7 (depends on 1)

The computing device of claim 1, wherein the contextual data is generated, at least in part, by an application configured to determine a priority, wherein the selection of the spatialization technology is further based on the priority.

### Claim 8

A computer-implemented method, comprising:
receiving, at a computing device, contextual data indicating capabilities of an encoder or one or more endpoint devices;
selecting, at the computing device, a spatialization technology based, at least in part, on the contextual data indicating capabilities of an encoder or one or more endpoint devices;
causing the encoder to generate a rendered output signal based on an input signal comprising object-based audio and channel-based audio processed by the selected spatialization technology; and
causing a communication of the rendered output signal from the encoder to the one or more endpoint devices.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the computer-implemented method further comprises:
receiving updated contextual data comprising one or more preferences, data indicating updated capabilities of an encoder, or data indicating updated capabilities of one or more endpoint devices; and
selecting, at the computing device, a second spatialization technology as the selected spatialization technology based, at least in part, on the updated contextual data.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, wherein the contextual data comprises one or more preferences prioritizing a plurality of spatialization technologies, including a first spatialization technology as a first priority and a second spatialization technology as a second priority, and wherein selecting the spatialization technology comprises:
determining when the encoder and the one or more endpoint devices is compatible with the first spatialization technology;
determining the first spatialization technology as the selected spatialization technology when the encoder and the one or more endpoint devices is compatible with the first spatialization technology;
determining when the encoder and the one or more endpoint devices is compatible with the second spatialization technology;
determining the second spatialization technology as the selected spatialization technology when the encoder and the one or more endpoint devices is compatible with the second spatialization technology, and when the encoder or the one or more endpoint devices is not compatible with the first spatialization technology.

### Claim 11 (depends on 8)

The computer-implemented method of claim 8, wherein the contextual data comprises one or more preferences prioritizing a plurality of endpoint devices, including a first endpoint device as a first priority and a second endpoint device as a second priority, and wherein selecting the spatialization technology comprises:
determining that the first endpoint device of the one or more endpoint devices is compatible with the first spatialization technology;
determining when the first endpoint device is in communication with the encoder;
determining the first spatialization technology as the selected spatialization technology when it is determined that the first endpoint device is in communication with the encoder;
determining that the second endpoint device of the one or more endpoint devices is compatible with the second spatialization technology;
determining when the second endpoint device is in communication with the encoder; and
determining the second spatialization technology as the selected spatialization technology when it is determined that the second endpoint device is in communication with the encoder and when the first endpoint device is not in communication with the encoder.

### Claim 12 (depends on 8)

The computer-implemented method of claim 8, wherein selecting the spatialization technology comprises:
determining, based at least in part by the contextual data, that a first endpoint device of the one or more endpoint devices is compatible with a first spatialization technology;
determining when the first endpoint device is in communication with the encoder;
determining the first spatialization technology as the selected spatialization technology when it is determined that the first endpoint device is in communication with the encoder;
determining, based at least in part by the contextual data, that a second endpoint device of the one or more endpoint devices is compatible with a second spatialization technology;
determining when the second endpoint device is in communication with the encoder; and
determining the second spatialization technology as the selected spatialization technology when it is determined that the second endpoint device is in communication with the encoder.

### Claim 13 (depends on 8)

The computer-implemented method of claim 8, wherein the contextual data is generated, at least in part, by an application configured to receive an input, wherein the selection of the spatialization technology is further based on the input.

### Claim 14 (depends on 8)

The computer-implemented method of claim 8, wherein the contextual data is generated, at least in part, by an application configured to determine a priority, wherein the selection of the spatialization technology is further based on the priority.

### Claim 15

A computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by one or more processors of a computing device, cause the one or more processors of the computing device to:
receive contextual data indicating capabilities of an encoder or one or more endpoint devices;
select a spatialization technology based, at least in part, on the contextual data indicating capabilities of an encoder or one or more endpoint devices;
cause the encoder to generate a rendered output signal based on an input signal comprising object-based audio and channel-based audio processed by the selected spatialization technology; and
cause a communication of the rendered output signal from the encoder to the one or more endpoint devices.

### Claim 16 (depends on 15)

The computer-readable storage medium of claim 15, wherein the contextual data comprises one or more preferences, and wherein the selection of the spatialization technology is further based on the one or more preferences.

### Claim 17 (depends on 15)

The computer-readable storage medium of claim 15, wherein the contextual data comprises one or more preferences prioritizing a plurality of spatialization technologies, including a first spatialization technology as a first priority and a second spatialization technology as a second priority, and wherein selecting the spatialization technology comprises:
determining when the encoder and the one or more endpoint devices is compatible with the first spatialization technology;
determining the first spatialization technology as the selected spatialization technology when the encoder and the one or more endpoint devices is compatible with the first spatialization technology;
determining when the encoder and the one or more endpoint devices is compatible with the second spatialization technology;
determining the second spatialization technology as the selected spatialization technology when the encoder and the one or more endpoint devices is compatible with the second spatialization technology, and when the encoder or the one or more endpoint devices is not compatible with the first spatialization technology.

### Claim 18 (depends on 15)

The computer-readable storage medium of claim 15, wherein the contextual data comprises one or more preferences prioritizing a plurality of endpoint devices, including a first endpoint device as a first priority and a second endpoint device as a second priority, and wherein selecting the spatialization technology comprises:
determining that the first endpoint device of the one or more endpoint devices is compatible with the first spatialization technology;
determining when the first endpoint device is in communication with the encoder;
determining the first spatialization technology as the selected spatialization technology when it is determined that the first endpoint device is in communication with the encoder;
determining that the second endpoint device of the one or more endpoint devices is compatible with the second spatialization technology;
determining when the second endpoint device is in communication with the encoder; and
determining the second spatialization technology as the selected spatialization technology when it is determined that the second endpoint device is in communication with the encoder and when the first endpoint device is not in communication with the encoder.

### Claim 19 (depends on 15)

The computer-readable storage medium of claim 15, wherein selecting the spatialization technology comprises:
determining, based at least in part by the contextual data, that a first endpoint device of the one or more endpoint devices is compatible with a first spatialization technology;
determining when the first endpoint device is in communication with the encoder;
determining the first spatialization technology as the selected spatialization technology when it is determined that the first endpoint device is in communication with the encoder;
determining, based at least in part by the contextual data, that a second endpoint device of the one or more endpoint devices is compatible with a second spatialization technology;
determining when the second endpoint device is in communication with the encoder; and
determining the second spatialization technology as the selected spatialization technology when it is determined that the second endpoint device is in communication with the encoder.

### Claim 20 (depends on 15)

The computer-readable storage medium of claim 15, wherein the contextual data is generated, at least in part, by an application configured to receive an input, wherein the selection of the spatialization technology is further based on the input.
