# US 11,310,618 B2 · Virtual reality, augmented reality, and mixed reality systems with spatialized audio

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,310,618 B2
- Document ID: US11310618B2
- Status: Granted
- Kind: B2
- Application number: 16/560,670
- Filed: 2019-09-04
- Published: 2020-01-02
- Granted: 2022-04-19
- Earliest claimed: 2016-09-14
- Assignees: Magic Leap, Inc.
- Applicants: Magic Leap, Inc.
- Inventors: Gregory Michael Link
- Web page: https://patentgrep.com/patent/US11310618B2

## CPC

- H04S 7/303: {Tracking of listener position or orientation}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2400/15: Aspects of sound capture and related signal processing for recording or reproduction
- H04S: STEREOPHONIC SYSTEMS
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

A spatialized audio system includes a sensor to detect a head pose of a listener. The system also includes a processor to render audio data in first and second stages. The first stage includes rendering first audio data corresponding to a first plurality of sources to second audio data corresponding to a second plurality of sources. The second stage includes rendering the second audio data corresponding to the second plurality of sources to third audio data corresponding to a third plurality of sources based on the detected head pose of the listener. The second plurality of sources consists of fewer sources than the first plurality of sources.

## Claims

### Claim 1

A spatialized audio system, comprising:
a sensor to detect head pose of a listener; and
a processor to:
render, based on the detected head pose of the listener, first audio data corresponding to a first plurality of sources, each of the first plurality of sources having respective ones of a first plurality of positions, to second audio data corresponding to a second plurality of sources, each of the second plurality of sources having respective ones of a second plurality of positions; and
reproduce, based on the second audio data, a spatialized sound field corresponding to the first audio data for the listener,
wherein the first plurality of sources are virtual sound sources,
wherein the second plurality of sources are virtual sound sources and consists of fewer sources than the first plurality of sources, and
wherein rendering the first audio data to the second audio data comprises each of the second plurality of sources recording virtual sound generated by the first plurality of sources at a respective one of the second plurality of positions.

### Claim 2 (depends on 1)

The system of claim 1, wherein rendering the first audio data to the second audio data comprises warping the first audio data to the second audio data.

### Claim 3 (depends on 1)

The system of claim 1, wherein the sensor detects the head pose of the listener immediately before the processor renders the first audio data to the second audio data.

### Claim 4 (depends on 1)

The system of claim 1, wherein the first audio data is a full audio stream data set.

### Claim 5 (depends on 1)

The system of claim 1, wherein the second plurality of sources consists of 8 or fewer sources.

### Claim 6 (depends on 1)

The system of claim 1, wherein each of the second plurality of positions is a different position.

### Claim 7 (depends on 1)

The system of claim 1, wherein each of the second plurality of positions is closer to the listener than each of the first plurality of positions.

### Claim 8 (depends on 1)

The system of claim 1, wherein the second plurality of positions are not located in a single plane.

### Claim 9 (depends on 1)

The system of claim 1, further comprising a plurality of speakers corresponding to the second plurality of sources to produce sound based on the second audio data.

### Claim 10 (depends on 9)

The system of claim 9, wherein each of the plurality of speakers corresponds to a respective source of the second plurality of sources at a respective one of the second plurality of positions.

### Claim 11 (depends on 1)

The system of claim 1, wherein rendering the first audio data to the second audio data is further based on respective positions of the first plurality of sources.

### Claim 12 (depends on 1)

The system of claim 1, wherein rendering the first audio data to the second audio data is more sensitive to rotation than translation of the listener.

### Claim 13 (depends on 1)

The system of claim 1, wherein rendering the first audio data to the second audio data is a rotation-only audio transformation.

### Claim 14 (depends on 1)

The system of claim 1, wherein the sensor detects a second head pose of the listener, and
wherein the processor is configured to render, based on the detected second head pose of the listener, the second audio data to third audio data corresponding to a third plurality of sources, each of the third plurality of sources having one of a third plurality of positions, and
wherein rendering the second audio data to the third audio data comprises each of the third plurality of sources recording virtual sound generated by the second plurality of sources at a respective one of the third plurality of positions.

### Claim 15 (depends on 14)

The system of claim 14, wherein rendering the second audio data to the third audio data comprises warping the second audio data to the third audio data.

### Claim 16 (depends on 14)

The system of claim 14, wherein the sensor detects the detected second head pose of the listener immediately before the processor renders the second audio data to the third audio data.

### Claim 17 (depends on 14)

The system of claim 14, wherein the third plurality of sources consists of fewer sources than the second plurality of sources.

### Claim 18 (depends on 14)

The system of claim 14, further comprising a plurality of speakers corresponding to the third plurality of sources to produce sound based on the third audio data.