# US 11,910,182 B2 · Method for processing an audio signal, signal processing unit, binaural renderer, audio encoder and audio decoder

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,910,182 B2
- Document ID: US11910182B2
- Status: Granted
- Kind: B2
- Application number: 17/882,357
- Filed: 2022-08-05
- Published: 2023-02-02
- Granted: 2024-02-20
- Earliest claimed: 2013-07-22
- Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Applicants: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Inventors: Simone Neukam; Jan Plogsties
- Web page: https://patentgrep.com/patent/US11910182B2

## CPC

- H04S 7/305: {Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence)}
- G10K 15/08: Arrangements for producing a reverberation or echo sound {(modifying acoustic properties to change reverberation time G10K11/002)}
- G10K 15/12: using electronic time-delay networks
- G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
- G10L 25/06: the extracted parameters being correlation coefficients
- H04S 7/30: {Control circuits for electronic adaptation of the sound field}
- H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
- H04S 2400/03: Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 (H04S2400/01 takes precedence)
- H04S 2400/13: Aspects of volume control, not necessarily automatic, in stereophonic sound systems
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04S: STEREOPHONIC SYSTEMS
- G10K: SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method for processing an audio signal in accordance with a room impulse response is described. The audio signal is processed with an early part of the room impulse response separate from a late reverberation of the room impulse response, wherein the processing of the late reverberation has generating a scaled reverberated signal, the scaling being dependent on the audio signal. The processed early part of the audio signal and the scaled reverberated signal are combined.

## Claims

### Claim 1

A method for processing an audio signal in accordance with a room impulse response, the method comprising:
applying the audio signal as an input signal to an early part processor and to a late reverberation processor;
receiving, by the early part processor, a direct sound and early reflections of the room impulse response and processing the audio signal using the direct sound and early reflections to obtain a processed audio signal;
receiving, by the late reverberation processor, predefined reverberator parameters and processing the audio signal using the predefined reverberator parameters to obtain a reverberated signal;
wherein processing the audio signal by the late reverberation processor comprises a scaling to obtain a scaled reverberated signal; and
combining the processed audio signal and the scaled reverberated signal,
wherein the scaling comprises:
setting a gain factor according to a predefined correlation measure of the audio signal, the predefined correlation measure having a fixed value determined empirically on the basis of an analysis of a plurality of audio signals, and applying the gain factor, or
obtaining a gain factor using a correlation analysis of the audio signal, and
applying the gain factor.

### Claim 2 (depends on 1)

The method of claim 1, wherein the scaling is dependent on the condition of the one or more input channels of the audio signal, wherein the condition of the one or more input channels of the audio signal comprises one or more of the number of input channels, the number of active input channels and the activity in the input channel.

### Claim 3 (depends on 1)

The method of claim 1, wherein the gain factor is determined based on the condition of one or more input channels of the audio signal.

### Claim 4 (depends on 3)

The method of claim 3, wherein the gain factor is determined as follows:
g=cu+ρ(cc−cu)
where
ρ=the predefined correlation measure for the audio signal,
cu, cc=factors indicative of the condition of the one or more input channels of the audio signal, with cu referring to totally uncorrelated channels, and cc relating to totally correlated channels.

### Claim 5 (depends on 4)

The method of claim 4, wherein cu and cc are determined as follows:
c
u
=
10
10
·
log
10
(
K
in
)
20
=
K
in
c
c
=
10
20
·
log
10
(
K
in
)
20
=
K
in
where
K
in
=
number
⁢
of
⁢
active
⁢
or
⁢
fixed
⁢
downmix
⁢
channels
.

### Claim 6 (depends on 1)

The method of claim 1, wherein the scaling comprises applying the gain factor before, during or after processing the audio signal using the predefined reverberator parameters.

### Claim 7 (depends on 1)

The method of claim 1, wherein the correlation analysis of the audio signal comprises determining for an audio frame of the audio signal the predefined correlation measure, and wherein the predefined correlation measure is calculated by combining correlation coefficients for a plurality of channel combinations of one audio frame, each audio frame comprising one or more time slots.

### Claim 8 (depends on 7)

The method of claim 7, wherein combining the correlation coefficients comprises averaging a plurality of the correlation coefficients of the audio frame.

### Claim 9 (depends on 7)

The method of claim 7, wherein determining the predefined correlation measure comprises:
(i) calculating an overall mean value for every channel of the audio frame,
(ii) calculating a zero-mean audio frame by subtracting the mean values from every channel,
(iii) calculating for a plurality of channel combinations the correlation coefficient, and
(iv) calculating the predefined correlation measure as a mean of the plurality of correlation coefficients.

### Claim 10 (depends on 7)

The method of claim 7, wherein the correlation coefficient for a channel combination is calculated as follows:
ρ
[
m
,
n
]
=
❘
"\[LeftBracketingBar]"
1
(
N
-
1
)
·
∑
i
∑
j
x
m
[
i
,
j
]
·
x
n
[
i
,
j
]
*
∑
j
σ
⁡
(
x
m
[
j
]
)
·
σ
⁡
(
x
n
[
j
]
)
❘
"\[RightBracketingBar]"
where
ρ
[
m
,
n
]
=
correlation
⁢
coefficient
,
σ
⁡
(
x
m
[
j
]
)
=
standard
⁢
deviation
⁢
across
⁢
one
⁢
time
⁢
slot
⁢
j
⁢
of
⁢
channel
⁢
m
,
σ
⁡
(
x
n
[
j
]
)
=
standard
⁢
deviation
⁢
across
⁢
one
⁢
time
⁢
slot
⁢
⁢
j
⁢
of
⁢
channel
⁢
n
,
x
m
,
x
n
=
zero
-
mean
⁢
variables
,
i
⁢
∀
[
1
,
N
]
=
frequency
⁢
bands
,
j
⁢
∀
[
1
,
M
]
=
time
⁢
slots
,
m
,
n
⁢
∀
[
1
,
K
]
=
channels
,
*
=
complex
⁢
conjugate
.

### Claim 11 (depends on 1)

The method of claim 1, comprising delaying the scaled reverberated signal to match a start of the scaled reverberated signal to the transition point from early reflections to a late reverberation in the room impulse response.

### Claim 12 (depends on 1)

The method of claim 1, wherein processing the audio signal by the late reverberation processor comprises downmixing the audio signal.

### Claim 13 (depends on 1)

A non-transitory digital storage medium having stored thereon a computer program with program code for carrying out the method of claim 1 when being executed by a computer.

### Claim 14

A signal processing unit, comprising:
an input for receiving an audio signal,
an early part processor receiving a direct sound and early reflections of the room impulse response and processing the audio signal using the direct sound and early reflections to obtain a processed audio signal,
a late reverberation processor receiving as input signal the audio signal, wherein the late reverberation processor is to receive predefined reverberator parameters and to process the audio signal using the predefined reverberator parameters to obtain a reverberated signal and to scale the reverberated signal to obtain a scaled reverberated signal; and
an output for combining the processed audio signal and the scaled reverberated signal into an output audio signal,
wherein processing the audio signal by the late reverberation processor comprises a scaling to obtain a scaled reverberated signal, and
wherein the scaling comprises:
setting a gain factor according to a predefined correlation measure of the audio signal, the predefined correlation measure having a fixed value determined empirically on the basis of an analysis of a plurality of audio signals, and applying the gain factor, or
obtaining a gain factor using a correlation analysis of the audio signal, and applying the gain factor.

### Claim 15 (depends on 14)

The signal processing unit of claim 14, wherein the late reverberation processor comprises:
a reverberator receiving the audio signal and generating a reverberated signal; and
a gain stage coupled to an input or to an output of the reverberator and controlled by the gain factor.

### Claim 16 (depends on 15)

The signal processing unit of claim 15, comprising a correlation analyzer generating the gain factor dependent on the audio signal.

### Claim 17 (depends on 15)

The signal processing unit of claim 15, further comprising at least one of:
a low pass filter coupled to the gain stage, and
a delay element coupled between the gain stage and an adder, the adder further coupled to the early part processor and the output.

### Claim 18 (depends on 14)

A binaural renderer, comprising the signal processing unit of claim 14.

### Claim 19 (depends on 14, 18)

An audio encoder for coding audio signals, comprising:
the signal processing unit of claim 14 or the binaural renderer of claim 18.

### Claim 20 (depends on 14, 18)

An audio decoder for decoding encoded audio signals, comprising:
the signal processing unit of claim 14 or the binaural renderer of claim 18.