# US 11,910,180 B2 · Audio processing method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,910,180 B2
- Document ID: US11910180B2
- Status: Granted
- Kind: B2
- Application number: 18/173,542
- Filed: 2023-02-23
- Published: 2023-06-22
- Granted: 2024-02-20
- Earliest claimed: 2018-08-20
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Cal Armstrong; Gavin Kearney; Bin Wang; Zexin Liu
- Web page: https://patentgrep.com/patent/US11910180B2

## CPC

- H04S 7/302: {Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence)}
- H04S 7/303: {Tracking of listener position or orientation}
- H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04S 2420/11: Application of ambisonics in stereophonic audio systems
- H04S: STEREOPHONIC SYSTEMS

## Abstract

An audio processing method includes processing, by M first virtual speakers, a to-be-processed audio signal to obtain M first audio signals; processing, by N second virtual speakers, the to-be-processed audio signal to obtain N second audio signals; obtain M first head-related transfer functions (HRTFs) centered at a left ear position and N second HRTFs centered at a right ear position; obtain a first target audio signal based on the M first audio signals and the M first HRTFs; and obtain a second target audio signal based on the N second audio signals and the N second HRTFs.

## Claims

### Claim 1

An audio processing method comprising:
receiving a bitstream;
decoding the bitstream to obtain a to-be-processed audio signal, wherein the to-be-processed audio signal is an Ambisonics signal;
processing, by M first virtual speakers, the to-be-processed audio signal to obtain M first audio signals, wherein the M first virtual speakers are in a one-to-one correspondence with the M first audio signals, and wherein M is a first positive integer;
obtaining M first head-related transfer functions (HRTFs), wherein the M first HRTFs are centered at a left ear position, and wherein the M first HRTFs are in a one-to-one correspondence with the M first virtual speakers; and
obtaining a first target audio signal based on the M first audio signals and the M first HRTFs.

### Claim 2 (depends on 1)

The audio processing method of claim 1, wherein obtaining the first target audio signal based on the M first audio signals and the M first HRTFs comprises:
convolving each of the M first audio signals with a corresponding first HRTF to obtain M first convolved audio signals; and
obtaining the first target audio signal based on the M first convolved audio signals.

### Claim 3 (depends on 1)

The audio processing method of claim 1, further comprising storing correspondences between a plurality of preset positions and a plurality of HRTFs, wherein obtaining the M first HRTFs comprises:
obtaining M first positions of the M first virtual speakers relative to a current left ear position; and
determining, based on the M first positions and the correspondences, that the M first HRTFs correspond to the M first positions.

### Claim 4 (depends on 1)

The audio processing method of claim 1, further comprising storing correspondences between a plurality of preset positions and a plurality of HRTFs, wherein obtaining the M first HRTFs comprises:
obtaining M third positions of the M first virtual speakers relative to a current head center, wherein each of the M third positions comprises a first azimuth and a first elevation of a first virtual speaker relative to the current head center, and wherein each of the M third positions further comprises a first distance between the current head center and the first virtual speaker;
determining M fourth positions based on the M third positions, wherein the M third positions are in a one-to-one correspondence with the M fourth positions, wherein each of the M fourth positions and a corresponding M third position comprise a same elevation and a same distance, wherein a difference between an azimuth in each of the M fourth positions and a first value is the first azimuth in the corresponding M third position, wherein the first value is a difference between a first included angle and a second included angle, wherein the first included angle is between a first straight line and a first plane, wherein the second included angle is between a second straight line and the first plane, wherein the first straight line passes through a current left ear position and a coordinate origin of a three-dimensional coordinate system, wherein the second straight line passes through the current head center and the coordinate origin, and wherein the first plane is defined by an X axis and a Z axis of the three-dimensional coordinate system; and
determining, based on each of the M fourth positions and the correspondences, that the M first HRTFs correspond to the M fourth positions.

### Claim 5 (depends on 1)

The audio processing method of claim 1, further comprising storing correspondences between a plurality of preset positions and a plurality of HRTFs, wherein obtaining the M first HRTFs comprises:
obtaining M third positions of the M first virtual speakers relative to a current head center, wherein each of the M third positions comprises a first azimuth and a first elevation of a first virtual speaker relative to the current head center, and wherein each of the M third positions further comprises a first distance between the current head center and the first virtual speaker;
determining M seventh positions based on the M third positions, wherein the M third positions are in a one-to-one correspondence with the M seventh positions, wherein each of the M seventh positions and a corresponding M third position comprise a same elevation and a same distance, and wherein a difference between an azimuth in each of the M seventh positions and a first preset value is the first azimuth in the corresponding M third position; and
determining, based on the M seventh positions and the correspondences, that the M first HRTFs correspond to the M seventh positions.

### Claim 6 (depends on 1)

The audio processing method of claim 1, wherein prior to obtaining the M first audio signals, the audio processing method further comprises:
obtaining a target virtual speaker group, wherein the target virtual speaker group comprises M target virtual speakers, and wherein the M target virtual speakers are in a one-to-one correspondence with the M first virtual speakers; and
determining M tenth positions of the M first virtual speakers relative to a coordinate origin of a three-dimensional coordinate system based on M ninth positions of the M target virtual speakers relative to the coordinate origin, wherein the M ninth positions are in a one-to-one correspondence with the M tenth positions, wherein each of the M tenth positions and a corresponding M ninth position comprise a same elevation and a same distance, and wherein a difference between a first azimuth in each of the M tenth positions and a second preset value is a second azimuth in the corresponding M ninth position, and wherein obtaining the M first audio signals comprises processing the to-be-processed audio signal based on the M tenth positions to obtain the M first audio signals.

### Claim 7

An audio processing apparatus comprising:
one or more processor; and
a memory configured to store computer executable instructions, wherein the computer executable instructions when executed by the one or more processors cause the audio processing apparatus to:
receive a bitstream;
decode the bitstream to obtain a to-be-processed audio signal, wherein the to-be-processed audio signal is an Ambisonics signal;
process, by M first virtual speakers, a to-be-processed audio signal to obtain M first audio signals, wherein the M first virtual speakers are in a one-to-one correspondence with the M first audio signals;
obtain M first head-related transfer functions (HRTFs), wherein the M first HRTFs are centered at a left ear position, and wherein the M first HRTFs are in a one-to-one correspondence with the M first virtual speakers; and
obtain a first target audio signal based on the M first audio signals and the M first HRTFs.

### Claim 8 (depends on 7)

The audio processing apparatus of claim 7, wherein execution of the computer executable instructions further causes the audio processing apparatus to:
convolve each of the M first audio signals with a corresponding first HRTF to obtain M first convolved audio signals; and
obtain the first target audio signal based on the M first convolved audio signals.

### Claim 9 (depends on 7)

The audio processing apparatus of claim 7, wherein execution of the computer executable instructions further causes the audio processing apparatus to:
store correspondences between a plurality of preset positions and a plurality of HRTFs;
obtain M first positions of the M first virtual speakers relative to a current left ear position; and
determine, based on the M first positions and correspondences, that the M first HRTFs correspond to the M first positions.

### Claim 10 (depends on 7)

The audio processing apparatus of claim 7, wherein execution of the computer executable instructions further causes the audio processing apparatus to:
store correspondences between a plurality of preset positions and a plurality of HRTFs;
obtain M third positions of the M first virtual speakers relative to a current head center, wherein each of the M third positions comprises a first azimuth and a first elevation of a first virtual speaker relative to the current head center, and wherein each of the M third positions further comprises a first distance between the current head center and the first virtual speaker;
determine M fourth positions based on the M third positions, wherein the M third positions are in a one-to-one correspondence with the M fourth positions, wherein each of the M fourth positions and a corresponding M third position comprise a same elevation and a same distance, wherein a difference between an azimuth in each of the M fourth positions and a first value is the first azimuth in the corresponding M third position, wherein the first value is a difference between a first included angle and a second included angle, wherein the first included angle is between a first straight line and a first plane, wherein the second included angle is between a second straight line and the first plane, wherein the first straight line passes through a current left ear position and a coordinate origin of a three-dimensional coordinate system, wherein the second straight line passes through the current head center and the coordinate origin, and wherein the first plane is defined by an X axis and a Z axis of the three-dimensional coordinate system; and
determine, based on the M fourth positions and correspondences, that the M first HRTFs correspond to the M fourth positions.

### Claim 11 (depends on 7)

The audio processing apparatus of claim 7, wherein execution of the computer executable instructions further causes the audio processing apparatus to:
store correspondences between a plurality of preset positions and a plurality of HRTFs;
obtain M third positions of the M first virtual speakers relative to a current head center, wherein each of the M third positions comprises a first azimuth and a first elevation of a first virtual speaker relative to the current head center, and wherein each of the M third positions comprises a first distance between the current head center and the first virtual speaker;
determine M seventh positions based on the M third positions, wherein the M third positions are in a one-to-one correspondence with the M seventh positions, wherein each of the M seventh positions and a corresponding M third position comprise a same elevation and a same distance, and wherein a difference between an azimuth in each of the M seventh position and a first preset value is the first azimuth in the corresponding M third position; and
determine, based on the M seventh positions and correspondences, that the M first HRTFs correspond to the M seventh positions.

### Claim 12 (depends on 7)

The audio processing apparatus of claim 7, wherein execution of the computer executable instructions further causes the audio processing apparatus to:
obtain a target virtual speaker group, wherein the target virtual speaker group comprises M target virtual speakers, and wherein the M target virtual speakers are in a one-to-one correspondence with the M first virtual speakers; and
determine M tenth positions of the M first virtual speakers relative to a coordinate origin of a three-dimensional coordinate system based on M ninth positions of the M target virtual speakers relative to the coordinate origin, wherein the M ninth positions are in a one-to-one correspondence with the M tenth positions, wherein each of the M tenth positions and a corresponding M ninth position comprise a same elevation and a same distance, and wherein a difference between a first azimuth in each of the M tenth positions and a second preset value is a second azimuth in the corresponding M ninth position, and wherein the at least one processor is configured to obtain the M first audio signals by processing the to-be-processed audio signal based on the M tenth positions.

### Claim 13

A non-transitory computer-readable storage medium storing computer instructions, that when executed by one or more processors of a system, cause the system to:
receive a bitstream;
decode the bitstream to obtain a to-be-processed audio signal, wherein the to-be-processed audio signal is an Ambisonics signal;
process a to-be-processed audio signal by M first virtual speakers to obtain M first audio signals, wherein the M first virtual speakers are in a one-to-one correspondence with the M first audio signals;
obtain M first head-related transfer functions (HRTFs), wherein the M first HRTFs are centered at a left ear position, and wherein the M first HRTFs are in a one-to-one correspondence with the M first virtual speakers; and
obtain a first target audio signal based on the M first audio signals and the M first HRTFs.

### Claim 14 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the computer instructions, when executed by the one or more processors of the system, further cause the system to:
convolve each of the M first audio signals with a corresponding first HRTF to obtain M first convolved audio signals; and
obtain the first target audio signal based on the M first convolved audio signals.

### Claim 15 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the computer instructions, when executed by the one or more processors of the system, further cause the system to:
obtain M first positions of the M first virtual speakers relative to a current left ear position; and
determine, based on the M first positions and correspondences between a plurality of preset positions and a plurality of HRTFs, that the M first HRTFs correspond to the M first positions.

### Claim 16 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the computer instructions, when executed by the one or more processors of the system, further cause the system to:
obtain M third positions of the M first virtual speakers relative to a current head center, wherein each of the M third positions comprises a first azimuth and a first elevation of a first virtual speaker relative to the current head center, and wherein each of the M third positions further comprises a first distance between the current head center and the first virtual speaker;
determine M fourth positions based on the M third positions, wherein the M third positions are in a one-to-one correspondence with the M fourth positions, wherein each of the M fourth positions and a corresponding M third position comprise a same elevation and a same distance, wherein a difference between an azimuth in each of the M fourth positions and a first value is a first azimuth in the corresponding M third position, wherein the first value is a difference between a first included angle and a second included angle, wherein the first included angle is between a first straight line and a first plane, wherein the second included angle is between a second straight line and the first plane, wherein the first straight line passes through a current left ear position and a coordinate origin of a three-dimensional coordinate system, wherein the second straight line passes through the current head center and the coordinate origin, and wherein the first plane is defined by an X axis and a Z axis of the three-dimensional coordinate system; and
determine, based on the M fourth positions and correspondences between a plurality of preset positions and a plurality of HRTFs, that the M first HRTFs correspond to the M fourth positions.

### Claim 17 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the computer instructions, when executed by the one or more processors of the system, further cause the system to:
obtain M third positions of the M first virtual speakers relative to a current head center, wherein each of the M third positions comprises a first azimuth and a first elevation of a first virtual speaker relative to the current head center, and wherein each of the M fourth positions further comprises a first distance between the current head center and the first virtual speaker;
determine M seventh positions based on the M third positions, wherein the M third positions are in a one-to-one correspondence with the M seventh positions, each of the M seventh positions and a corresponding M third position comprise a same elevation and a same distance, and a difference between an azimuth in each of the M seventh positions and a first preset value is the first azimuth in the corresponding M third position; and
determining, based on the M seventh positions and correspondences between a plurality of preset positions and a plurality of HRTFs, that the M first HRTFs correspond to the M seventh positions.

### Claim 18 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the computer instructions, when executed by the one or more processors of the system, further cause the system to:
obtain a target virtual speaker group, wherein the target virtual speaker group comprises M target virtual speakers, and wherein the M target virtual speakers are in a one-to-one correspondence with the M first virtual speakers; and
determine M tenth positions of the M first virtual speakers relative to a coordinate origin of a three-dimensional coordinate system based on M ninth positions of the M target virtual speakers relative to the coordinate origin, wherein the M ninth positions are in a one-to-one correspondence with the M tenth positions, wherein each of the M tenth positions and a corresponding M ninth position comprise a same elevation and a same distance, and wherein a difference between an azimuth in each of the M tenth positions and a second preset value is the azimuth in the corresponding M ninth position, and wherein the one or more processors are configured to obtain the M first audio signals by processing the to-be-processed audio signal based on the M tenth positions.

### Claim 19 (depends on 1)

The audio processing method of claim 1, further comprising:
processing, by N second virtual speakers, the to-be-processed audio signal to obtain N second audio signals, wherein the N second virtual speakers are in a one-to-one correspondence with the N second audio signals, and wherein N is a second positive integer;
obtaining N second HRTFs centered at a right ear position;
obtaining a second target audio signal based on the N second audio signals and the N second HRTFs;
transmitting the first target audio signal to a left ear; and
transmitting the second target audio signal to a right ear.

### Claim 20 (depends on 7)

The audio processing apparatus of claim 7, wherein execution of the computer executable instructions further causes the audio processing apparatus to:
process, by N second virtual speakers, the to-be-processed audio signal to obtain N second audio signals, wherein the N second virtual speakers are in a one-to-one correspondence with the N second audio signals, and wherein N is a second positive integer;
obtain N second HRTFs centered at a right ear position;
obtain a second target audio signal based on the N second audio signals and the N second HRTFs;
transmitting the first target audio signal to a left ear; and
transmitting the second target audio signal to a right ear.