# US 2022/0046375 A1 · Efficient rendering of virtual soundfields

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0046375 A1
- Document ID: US20220046375A1
- Status: Published application
- Kind: A1
- Application number: 17/412,084
- Filed: 2021-08-25
- Published: 2022-02-10
- Earliest claimed: 2018-06-12
- Assignees: Magic Leap, Inc.
- Applicants: Magic Leap, Inc.
- Inventors: Brian Lloyd Schmidt; Samuel Charles Dicker
- Web page: https://patentgrep.com/patent/US20220046375A1

## CPC

- H04R 1/1041: {Mechanical or electronic switches, or control elements (switches in general H01H)}
- H04S 7/303: {Tracking of listener position or orientation}
- G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
- G10L 25/21: the extracted parameters being power information
- H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- H04S 7/304: {For headphones}
- H04R 2460/07: Use of position data from wide-area or local-area positioning systems in hearing devices, e.g. program or information selection
- H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2400/13: Aspects of volume control, not necessarily automatic, in stereophonic sound systems
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- H04S: STEREOPHONIC SYSTEMS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An audio system and method of spatially rendering audio signals that uses modified virtual speaker panning is disclosed. The audio system may include a fixed number F of virtual speakers, and the modified virtual speaker panning may dynamically select and use a subset P of the fixed virtual speakers. The subset P of virtual speakers may be selected using a low energy speaker detection and culling method, a source geometry-based culling method, or both. One or more processing blocks in the decoder/virtualizer may be bypassed based on the energy level of the associated audio signal or the location of the sound source relative to the user/listener, respectively. In some embodiments, a virtual speaker that is designated as an active virtual speaker at a first time, may also be designated as an active virtual speaker at a second time to ensure the processing completes.

## Claims

### Claim 1

A method of spatially rendering an audio signal, the method comprising:
determining a model of a virtual environment;
determining a spatial configuration of the virtual environment, wherein the spatial configuration comprises at least a user location, a sound source location, and a virtual speaker location;
determining one or more signals associated with the spatial configuration and further associated with the user location, the sound source location, or the virtual speaker location;
determining whether one or more signals corresponding to the sound source in the virtual environment exceeds a predetermined threshold;
in accordance with a determination that the one or more signals exceeds a predetermined threshold, decoding the one or more signals; and
rendering the audio signal based on the one or more signals.

### Claim 2 (depends on 1)

The method of claim 1, wherein decoding the one or more signals comprises performing one or more processing blocks, and wherein the method further comprises:
selectively bypassing one or more of the processing blocks, the one or more of the processing blocks associated with one or more inactive virtual speakers.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
determining an energy level associated with the one or more signals;
determining whether the energy level is less than an energy threshold;
in accordance with a determination that the energy level is not less than the energy threshold, performing a head related transfer function (HRTF) processing of the one or more signals;
in accordance with a determination that the energy level is less than the energy threshold, forgoing performing the HRTF processing of the one or more signals.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
determining whether a number of sound sources in the virtual environment exceeds a predetermined threshold,
wherein the selective bypass of the one or more processing blocks includes bypassing a plurality of detectors in accordance with a determination that the number of sound sources exceeds the predetermined threshold.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
in accordance with a determination that the number of sound sources does not exceed the predetermined threshold, detecting an energy level of the one or more signals using the plurality of detectors.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
determining whether the energy level is less than an energy threshold;
in accordance with a determination that the energy level is not less than the energy threshold, performing a head related transfer function (HRTF) processing of the one or more signals;
in accordance with a determination that the energy level is less than the energy threshold, forgoing performing the HRTF processing of the one or more signals.

### Claim 7 (depends on 1)

The method of claim 1,
wherein, determining a model of a virtual environment further comprises:
receiving one or more sound signals from at least a direct sound source and reflection sound source;
modifying the one or more sound signals to simulate a doppler effect;
adding a delay to the one or more sound signals; and
panning the one or more sound signals across a plurality of virtual speakers, and wherein, decoding the one or more signals further comprises:
determining one or more virtualized sounds associated with a movement of a sound source, a user, or both.

### Claim 8

A system to spatially render an audio signal, the system comprising:
a wearable head device configured to provide the audio signal to a user; and
one or more processors configured to execute a method comprising:
determining a model of a virtual environment;
determining a spatial configuration of the virtual environment, wherein the spatial configuration comprises at least a user location, a sound source location, and a virtual speaker location;
determining one or more signals associated with the spatial configuration and further associated with one or more of the user location, the sound source location, or the virtual speaker location;
determining whether one or more signals corresponding to the sound source in the virtual environment exceeds a predetermined threshold;
in accordance with a determination that the one or more signals exceeds a predetermined threshold, decoding the one or more signals; and
rendering the audio signal based on the one or more signals.

### Claim 9 (depends on 8)

The system of claim 8, wherein decoding the one or more signals comprises performing one or more processing blocks, and wherein the method further comprises:
selectively bypassing one or more of the processing blocks, the one or more of the processing blocks associated with one or more inactive virtual speakers.

### Claim 10 (depends on 8)

The system of claim 8, wherein the method further comprises:
determining an energy level associated with the one or more signals;
determining whether the energy level is less than an energy threshold;
in accordance with a determination that the energy level is not less than the energy threshold, performing a head related transfer function (HRTF) processing of the one or more signals;
in accordance with a determination that the energy level is less than the energy threshold, forgoing performing the HRTF processing of the one or more signals.

### Claim 11 (depends on 9)

The system of claim 9, wherein the method further comprises:
determining whether a number of sound sources in the virtual environment exceeds a predetermined threshold,
wherein the selective bypass of the one or more processing blocks includes bypassing a plurality of detectors in accordance with a determination that the number of sound sources exceeds the predetermined threshold.

### Claim 12 (depends on 11)

The system of claim 11, wherein the method further comprises:
in accordance with a determination that the number of sound sources does not exceed the predetermined threshold, detecting an energy level of the one or more signals using the plurality of detectors.

### Claim 13 (depends on 12)

The system of claim 12, wherein the method further comprises:
determining whether the energy level is less than an energy threshold;
in accordance with a determination that the energy level is not less than the energy threshold, performing a head related transfer function (HRTF) processing of the one or more signals;
in accordance with a determination that the energy level is less than the energy threshold, forgoing performing the HRTF processing of the one or more signals.

### Claim 14 (depends on 8)

The system of claim 8,
wherein, determining a model of a virtual environment further comprises:
receiving one or more sound signals from at least a direct sound source and a reflection sound source;
modifying the one or more sound signals to simulate a doppler effect;
adding a delay to the one or more sound signals; and
panning the one or more sound signals across a plurality of virtual speakers, and wherein, decoding the one or more signals further comprises:
determining one or more virtualized sounds associated with a movement of a sound source, a user, or both.
