# US 2021/0084424 A1 · Method for generating binaural signals from stereo signals using upmixing binauralization, and apparatus therefor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0084424 A1
- Document ID: US20210084424A1
- Status: Published application
- Kind: A1
- Application number: 17/022,065
- Filed: 2020-09-15
- Published: 2021-03-18
- Earliest claimed: 2019-09-16
- Assignees: Gaudio Lab, Inc.
- Applicants: Gaudio Lab, Inc.
- Inventors: Sangbae CHON; Byoungjoon AHN; Jaesung CHOI; Hyunoh OH; Jeonghun SEO; Taegyu LEE
- Web page: https://patentgrep.com/patent/US20210084424A1

## CPC

- H04S 1/002: {Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution (control circuits for electronic adaptation of the sound field H04S7/30)}
- H04S 7/302: {Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence)}
- H04S 1/007: {in which the audio signals are in digital form (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- H04S 7/30: {Control circuits for electronic adaptation of the sound field}
- H04S 5/005: {of the pseudo five- or more-channel type, e.g. virtual surround}
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04S: STEREOPHONIC SYSTEMS

## Abstract

Disclosed is an audio signal processing method including: receiving a stereo signal; transforming the stereo signal into a frequency-domain signal; rendering the first signal based on a first ipsilateral filter coefficient; generating a frontal ipsilateral signal relating to the frequency-domain signal; rendering the second signal based on a second ipsilateral filter coefficient; generating a side ipsilateral signal relating to the frequency-domain signal; rendering the second signal based on a contralateral filter coefficient; generating a side contralateral signal relating to the frequency-domain signal; transforming an ipsilateral signal, generated by mixing the frontal ipsilateral signal and the side ipsilateral signal, and the side contralateral signal into a time-domain ipsilateral signal and a time-domain contralateral signal, which are time-domain signals, respectively; and generating a binaural signal by mixing the time-domain ipsilateral signal and the time-domain contralateral signal.

## Claims

### Claim 1

An audio signal processing method comprising:
receiving a stereo signal;
transforming the stereo signal into a frequency-domain signal;
separating the frequency-domain signal into a first signal and a second signal based on an inter-channel correlation and an inter-channel level difference (ICLD) of the frequency-domain signal, wherein the first signal includes a frontal component of the frequency-domain signal, and the second signal includes a side component of the frequency-domain signal;
rendering the first signal based on a first ipsilateral filter coefficient and generating a frontal ipsilateral signal relating to the frequency-domain signal, wherein the first ipsilateral filter coefficient is generated based on an ipsilateral response signal of a first head-related impulse response (HRIR);
rendering the second signal based on a second ipsilateral filter coefficient and generating a side ipsilateral signal relating to the frequency-domain signal, wherein the second ipsilateral filter coefficient is generated based on an ipsilateral response signal of a second HRIR;
rendering the second signal based on a contralateral filter coefficient and generating a side contralateral signal relating to the frequency-domain signal, wherein the contralateral filter coefficient is generated based on a contralateral response signal of the second HRIR;
transforming an ipsilateral signal, generated by mixing the frontal ipsilateral signal and the side ipsilateral signal, and the side contralateral signal into a time-domain ipsilateral signal and a time-domain contralateral signal, which are time-domain signals, respectively; and
generating a binaural signal by mixing the time-domain ipsilateral signal and the time-domain contralateral signal,
wherein the binaural signal is generated in consideration of an interaural time delay (ITD) applied to the time-domain contralateral signal, and
wherein the first ipsilateral filter coefficient, the second ipsilateral filter coefficient, and the contralateral filter coefficient are real numbers.

### Claim 2 (depends on 1)

The method of claim 1, wherein a sum of a left-channel signal of the first signal and a left-channel signal of the second signal is the same as a left-channel signal of the stereo signal.

### Claim 3 (depends on 1)

The method of claim 1, wherein a sum of a right-channel signal of the first signal and a right-channel signal of the second signal is the same as a right-channel signal of the stereo signal.

### Claim 4 (depends on 1)

The method of claim 1, wherein energy of a left-channel signal of the first signal and energy of a right-channel signal of the first signal are the same as each other.

### Claim 5 (depends on 1)

The method of claim 1, wherein a contralateral characteristic of the HRIR in consideration of ITD is applied to an ipsilateral characteristic of the HRIR.

### Claim 6 (depends on 1)

The method of claim 1, wherein the ITD is 1 ms or less.

### Claim 7 (depends on 1)

The method of claim 1, wherein a phase of a left-channel signal of the first signal is the same as a phase of the left-channel signal of the frontal ipsilateral signal,
wherein a phase of a right-channel signal of the first signal is the same as a phase of a right-channel signal of the frontal ipsilateral signal,
wherein a phase of a left-channel signal of the second signal, a phase of a left-side signal of the side ipsilateral signal, and a phase of a left-side signal of the side contralateral signal are the same, and
wherein a phase of a right-channel signal of the second signal, a phase of a right-side signal of the side ipsilateral signal, and a phase of a right-side signal of the side contralateral signal are the same.

### Claim 8 (depends on 1)

The method of claim 1, wherein the transforming of the ipsilateral signal, generated by mixing the frontal ipsilateral signal and the side ipsilateral signal, and the side contralateral signal into a time-domain ipsilateral signal and a time-domain contralateral signal, which are time-domain signals, respectively, comprises:
transforming a left ipsilateral signal and a right ipsilateral signal, generated by mixing a frontal ipsilateral signal and a side ipsilateral signal for each of left and right channels, into a time-domain left ipsilateral signal and a time-domain right ipsilateral signal, which are time-domain signals, respectively; and
transforming the side contralateral signal into a left-side contralateral signal and a right-side contralateral signal, which are time-domain signals, for each of left and right channels,
wherein the binaural signal is generated by mixing the time-domain left ipsilateral signal and a time-domain left-side contralateral signal and mixing the time-domain right ipsilateral signal and a time-domain right-side contralateral signal.

### Claim 9

An audio signal processing apparatus comprising:
an input terminal configured to receive a stereo signal; and
a processor including a renderer,
wherein the processor is configured to:
transform the stereo signal into a frequency-domain signal;
separate the frequency-domain signal into a first signal and a second signal based on an inter-channel correlation and an inter-channel level difference (ICLD) of the frequency-domain signal, wherein the first signal includes a frontal component of the frequency-domain signal, and the second signal includes a side component of the frequency-domain signal;
render the first signal based on a first ipsilateral filter coefficient and generate a frontal ipsilateral signal relating to the frequency-domain signal, wherein the first ipsilateral filter coefficient is generated based on an ipsilateral response signal of a first head-related impulse response (HRIR);
render the second signal based on a second ipsilateral filter coefficient and generate a side ipsilateral signal relating to the frequency-domain signal, wherein the second ipsilateral filter coefficient is generated based on an ipsilateral response signal of a second HRIR;
render the second signal based on a contralateral filter coefficient and generate a side contralateral signal relating to the frequency-domain signal, wherein the contralateral filter coefficient is generated based on a contralateral response signal of the second HRIR;
transform an ipsilateral signal, generated by mixing the frontal ipsilateral signal and the side ipsilateral signal, and the side contralateral signal into a time-domain ipsilateral signal and a time-domain contralateral signal, which are time-domain signals, respectively; and
generate a binaural signal by mixing the time-domain ipsilateral signal and the time-domain contralateral signal,
wherein the binaural signal is generated in consideration of an interaural time delay (ITD) applied to the time-domain contralateral signal, and
wherein the first ipsilateral filter coefficient, the second ipsilateral filter coefficient, and the contralateral filter coefficient are real numbers.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein a sum of a left-channel signal of the first signal and a left-channel signal of the second signal is the same as a left-channel signal of the stereo signal.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein a sum of a right-channel signal of the first signal and a right-channel signal of the second signal is the same as a right-channel signal of the stereo signal.

### Claim 12 (depends on 9)

The apparatus of claim 9, wherein energy of the left-channel signal of the first signal and energy of the right-channel signal of the first signal are the same as each other.

### Claim 13 (depends on 9)

The apparatus of claim 9, wherein a contralateral characteristic of the HRIR in consideration of ITD is applied to an ipsilateral characteristic of the HRIR.

### Claim 14 (depends on 9)

The apparatus of claim 9, wherein the ITD is 1 ms or less.

### Claim 15 (depends on 9)

The apparatus of claim 9, wherein a phase of the left-channel signal of the first signal is the same as a phase of the left-channel signal of the frontal ipsilateral signal,
wherein a phase of the right-channel signal of the first signal is the same as a phase of the right-channel signal of the frontal ipsilateral signal,
wherein a phase of the left-channel signal of the second signal, a phase of a left-side signal of the side ipsilateral signal, and a phase of a left-side signal of the contralateral signal are the same, and
wherein a phase of a right-channel signal of the second signal, a phase of a right-side signal of the side ipsilateral signal, and a phase of a right-side signal of the side contralateral signal are the same.

### Claim 16 (depends on 9)

The apparatus of claim 9, wherein the transforming, by the processor, of the ipsilateral signal, generated by mixing the frontal ipsilateral signal and the side ipsilateral signal, and the side contralateral signal into a time-domain ipsilateral signal and a time-domain contralateral signal, which are time-domain signals, respectively, comprises:
transforming a left ipsilateral signal and a right ipsilateral signal, generated by mixing the frontal ipsilateral signal and the side ipsilateral signal for each of left and right channels, into a time-domain left ipsilateral signal and a time-domain right ipsilateral signal, which are time-domain signals, respectively; and
transforming the side contralateral signal into a left-side contralateral signal and a right-side contralateral signal, which are time-domain signals, for each of left and right channels,
wherein the binaural signal is generated by mixing the time-domain left ipsilateral signal and a time-domain left-side contralateral signal and mixing the time-domain right ipsilateral signal and a time-domain right-side contralateral signal.
