# US 10,319,394 B2 · Apparatus and method for improving speech intelligibility in background noise by amplification and compression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,319,394 B2
- Document ID: US10319394B2
- Status: Granted
- Kind: B2
- Application number: 14/794,629
- Filed: 2015-07-08
- Published: 2015-10-29
- Granted: 2019-06-11
- Earliest claimed: 2013-01-08
- Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Applicants: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Inventors: Jan Rennies; Henning Schepker; Simon Doclo; Jens E. Appell
- Web page: https://patentgrep.com/patent/US10319394B2

## CPC

- G10L 21/0364: for improving intelligibility
- G10L 21/0316: by changing the amplitude
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An apparatus for generating a modified speech signal from a speech input signal which has a plurality of speech subband signals, the modified speech signal having a plurality of modified subband signals is provided, having: a weighting information generator for generating weighting information for each speech subband signal depending on a signal power of said speech subband signal, and a signal modifier for modifying each speech subband signal by applying the weighting information on said speech subband signal to obtain a modified subband signal. The weighting information generator is configured to generate the weighting information for each of the plurality of speech subband signals, wherein the signal modifier is configured to modify each of the speech subband signals so that a first speech subband signal having a first signal power is amplified with a first degree, and so that a second speech subband signal having a second signal power is amplified with a second degree, the first signal power being greater than the second signal power, and the first degree being lower than the second degree.

## Claims

### Claim 1

An apparatus for generating a modified audio speech signal from an audio speech input signal, wherein the audio speech input signal comprises a plurality of speech subband signals, wherein the modified speech signal comprises a plurality of modified subband signals, wherein the apparatus comprises:
a weighting information generator for generating weighting information for each speech subband signal of the plurality of speech subband signals depending on a signal power of said speech subband signal, and
a signal modifier for modifying each speech subband signal of the plurality of speech subband signals by applying the weighting information of said speech subband signal on said speech subband signal to acquire a modified subband signal of the plurality of modified subband signals,
wherein the apparatus is configured to output the modified audio speech signal,
wherein the weighting information generator is configured to generate the weighting information for each of the plurality of speech subband signals and wherein the signal modifier is configured to modify each of the speech subband signals so that a first speech subband signal of the plurality of speech subband signals comprising a first signal power is amplified with a first degree, and so that a second speech subband signal of the plurality of speech subband signals comprising a second signal power is amplified with a second degree, wherein the first signal power is greater than the second signal power, and wherein the first degree is lower than the second degree,
wherein the apparatus is implemented using a hardware apparatus or a computer or a combination of a hardware apparatus and a computer.

### Claim 2 (depends on 1)

The apparatus according to claim 1,
wherein a noise subband signal of a plurality of noise subband signals of a noise input signal is assigned to each speech subband signal of the plurality of speech subband signals, and
wherein the weighting information generator is configured to generate the weighting information of each speech subband signal of the plurality of speech subband signals depending on a noise spectrum level of the noise subband signal of said speech subband signal, and
wherein the weighting information generator is configured to generate the weighting information of each speech subband signal of the plurality of speech subband signals depending on a speech spectrum level of said speech subband signal.

### Claim 3 (depends on 2)

The apparatus according to claim 2, wherein the weighting information generator is configured to generate the weighting information of each speech subband signal of the plurality of speech subband signals by determining a signal-to-noise ratio of said speech spectrum level of said speech subband signal and of said noise spectrum level of the noise subband signal of said speech subband signal.

### Claim 4 (depends on 3)

The apparatus according to claim 3, wherein the signal-to-noise ratio q(en, dn) of said speech spectrum level of said speech subband signal and of said noise spectrum level of the noise subband signal of said speech subband signal is defined according to the formula
q
⁡
(
e
n
,
d
n
)
=
{
0
if
⁢
⁢
e
n
≤
d
n
-
15
⁢
⁢
dB
e
n
-
d
n
+
15
⁢
⁢
dB
30
⁢
⁢
dB
if
⁢
⁢
d
n
-
15
⁢
⁢
dB
<
e
n
≤
d
n
+
15
⁢
⁢
dB
1
if
⁢
⁢
e
n
>
d
n
+
15
⁢
⁢
dB
wherein en is said speech spectrum level of said speech subband signal, and
wherein dn is said noise spectrum level of the noise subband signal of said speech subband signal.

### Claim 5 (depends on 3)

The apparatus according to claim 3,
wherein the weighting information generator is configured to generate the weighting information of the plurality of speech subband signals of the audio speech input signal by determining a speech intelligibility index and by determining for each speech subband signal of the plurality of speech subband signal a signal-to-noise ratio of the speech spectrum level of said speech subband signal and of said noise spectrum level of the noise subband signal of said speech subband signal,
wherein the speech intelligibility index indicates a speech intelligibility of the audio speech input signal.

### Claim 6 (depends on 5)

The apparatus according to claim 5,
wherein the weighting information generator is configured to determine the speech intelligibility index {tilde over (S)}II[l] according to the formula
SII
~
⁡
[
l
]
=
∑
n
=
1
N
⁢
i
n
·
q
⁡
(
e
n
⁡
[
l
]
,
d
n
⁡
[
l
]
)
·
min
⁢
{
1
-
d
n
⁡
[
l
]
+
15
⁢
⁢
dB
-
u
n
-
10
⁢
⁢
dB
160
⁢
⁢
dB
,
1
}
,
wherein n indicates the n-th speech subband signal of the plurality of speech subband signals, wherein N indicates the total number of speech subband signals, wherein l indicates a block, wherein q(en, dn) indicates the signal-to-noise ratio of said speech spectrum level of the n-th speech subband signal and of said noise spectrum level of the noise subband signal of the n-th speech subband signal, wherein un indicates a speech spectrum level being a fixed value, and wherein in indicates a band importance.

### Claim 7 (depends on 5)

The apparatus according to claim 5, wherein the weighting information generator is configured to generate the weighting information of each speech subband signal of the plurality of speech subband signals by determining a linear gain for each speech subband signal of the plurality of speech subband signals depending on the speech intelligibility index, depending on the signal power of said speech subband signal and depending on the sum of the signal powers of all speech subband signals of the plurality of speech subband signals.

### Claim 8 (depends on 7)

The apparatus according to claim 7, wherein the weighting information generator is configured to generate a linear gain wn,(lin) for each speech subband signal of the plurality of speech subband signals according to the formula
w
n
,
(
lin
)
⁡
[
l
]
=
ϕ
n
SII
~
⁡
[
l
]
∑
λ
=
1
N
⁢
ϕ
λ
SII
~
⁡
[
l
]
·
ϕ
(
max
)
⁡
[
l
]
ϕ
n
⁡
[
l
]
wherein n indicates the n-th speech subband signal of the plurality of speech subband signals, wherein N indicates the total number of speech subband signals, wherein l indicates a block, wherein Φn [l] indicates the signal power of the n-th speech subband signal, and wherein Φ(max) [l] is the sum of the signal powers of all speech subband signals of the plurality of speech subband signals.

### Claim 9 (depends on 3)

The apparatus according to claim 3,
wherein the weighting information generator is configured to determine a compression ratio crn [l] according to the formula
crn[l]=max{cr(max)·(1−q(en[l],dn[l])),1}.
wherein q(en[l], dn[l]) is the signal-to-noise ratio of said speech spectrum level, wherein the signal-to-noise ratio q(en[l], dn[l]) indicates a number between 0 and 1, wherein cr(max) indicates a fixed number, and wherein l indicates a block.

### Claim 10 (depends on 7)

The apparatus according to claim 7,
wherein the weighting information generator is configured to determine a compression ratio crn [l] according to the formula
crn[l]=max{cr(max)·(1−q(en[l],dn[l])),1}.
wherein q(en[l], dn[l]) is the signal-to-noise ratio of said speech spectrum level, wherein the signal-to-noise ratio q(en[l], dn[l]) indicates a number between 0 and 1, wherein cr(max) indicates a fixed number, and wherein l indicates a block.

### Claim 11 (depends on 9)

The apparatus according to claim 9,
wherein the weighting information generator is configured to generate the weighting information of each speech subband signal of the plurality of speech subband signals by determining a compressive gain wn,(comp) of said subband signal according to the formula
w
n
,
(
comp
)
⁡
[
l
·
M
-
m
]
=
(
ϕ
n
⁡
[
l
]
s
^
n
2
⁡
[
l
·
M
-
m
]
)
(
cr
n
⁡
[
l
]
-
1
)
cr
n
⁡
[
l
]
,
⁢
m
=
0
,
…
⁢
,
M
-
1
,
wherein M indicates a length of the block l, wherein Φn [l] indicates the signal power of said speech subband signal, and wherein ŝn2[l·M−m] indicates a square of a smoothed estimate of an envelope of a speech signal amplitude of said speech subband signal.

### Claim 12 (depends on 11)

The apparatus according to claim 11,
wherein the weighting information generator is configured to determine the smoothed estimate ŝ[k] of the envelope of the speech signal amplitude of said speech subband signal according to the formula
s
^
n
⁡
[
k
]
=
{
s
^
n
⁡
[
k
-
1
]
·
α
a
+
(
1
-
α
a
)
·

s
n
⁡
[
k
]

if
⁢
⁢

s
n
⁡
[
k
]

≥
s
^
n
⁡
[
k
-
1
]
s
^
n
⁡
[
k
-
1
]
·
α
r
+
(
1
-
α
r
)
·

s
n
⁡
[
k
]

if
⁢
⁢

s
n
⁡
[
k
]

<
s
^
n
⁡
[
k
-
1
]
wherein sn [k] indicates said speech subband signal, wherein |sn [k]| indicates the amplitude of said speech subband signal, wherein αa is a first smoothing constant and wherein αr is a second smoothing constant.

### Claim 13 (depends on 1)

The apparatus according to claim 1, wherein the weighting information generator is configured to generate the weighting information wn of each speech subband signal of the plurality of speech subband signals by applying the formula
wn[l·M−m]=αpwn[l·M−m−1]+(1−αp)pλn[l](ŝn2[l·M−m])
wherein n indicates the n-th speech subband signal of the plurality of speech subband signals, wherein N indicates the total number of speech subband signals, wherein l indicates a block, wherein αp is a smoothing constant, and wherein ŝn2[l·M−M] indicates a square of a smoothed estimate of an envelope of a speech signal amplitude of said speech subband signal, wherein pλn[l](ŝn2[l·M−m]) indicates a function that performs linear interpolation and extrapolation of λn[l] wherein λn[l] indicates a smoothed input-output characteristic.

### Claim 14 (depends on 1)

The apparatus according to claim 1, wherein the weighting information generator is configured to generate the weighting information for each of the plurality of speech subband signals and wherein the signal modifier is configured to modify each of the speech subband signals so that a first sum of all speech signal powers of all speech subband signals varies by less than 20% from a second sum of all speech signals powers of all modified subband signals.

### Claim 15 (depends on 2)

The apparatus according to claim 2, wherein the weighting information generator is configured to generate the weighting information of each speech subband signal of the plurality of speech subband signals by determining a weighted addition, wherein the weighted addition depends on the noise spectrum level of the noise subband signal of said speech subband signal and depends on a reverberation spectrum level.

### Claim 16 (depends on 15)

The apparatus according to claim 15, wherein the weighting information generator is configured to generate the reverberation spectrum level depending on a room impulse response between a loudspeaker and a microphone, depending on a reverberation time T60 or depending on a direct-to-reverberation energy ratio.

### Claim 17 (depends on 15)

The apparatus according to claim 15, wherein the weighting information generator is configured to determine the weighted addition an [l] according to the formula
an[l]=βzn[l]+dn[l],
wherein dn [l] is said noise spectrum level of the noise subband signal of said speech subband signal, wherein zn [l] indicates said reverberation spectrum level, and wherein β is a real value.

### Claim 18 (depends on 1)

The apparatus according to claim 1, wherein the apparatus further comprises a first filterbank and a second filterbank,
wherein the first filterbank is configured to transform an unprocessed speech signal, being represented in a time domain, from the time domain to a subband domain to acquire the audio speech input signal comprising the plurality of speech subband signals, and
wherein the second filterbank is configured to transform the modified audio speech signal, being represented in the subband domain and comprising the plurality of modified subband signals, from the subband domain to the time domain to acquire a time-domain output signal.

### Claim 19

A method for generating a modified audio speech signal from an audio speech input signal, wherein the audio speech input signal comprises a plurality of speech subband signals, wherein the modified audio speech signal comprises a plurality of modified subband signals, wherein the method comprises:
generating weighting information for each speech subband signal of the plurality of speech subband signals depending on a signal power of said speech subband signal,
modifying each speech subband signal of the plurality of speech subband signals by applying the weighting information of said speech subband signal on said speech subband signal to acquire a modified subband signal of the plurality of modified subband signals, and
outputting the modified audio speech signal,
wherein generating the weighting information for each of the plurality of speech subband signals and modifying each of the speech subband signals are conducted so that a first speech subband signal of the plurality of speech subband signals comprising a first signal power is amplified with a first degree, and so that a second speech subband signal of the plurality of speech subband signals comprising a second signal power is amplified with a second degree, wherein the first signal power is greater than the second signal power, and wherein the first degree is lower than the second degree,
wherein the method is performed using a hardware apparatus or a computer or a combination of a hardware apparatus and a computer.

### Claim 20

A non-transitory computer-readable medium comprising a computer program for implementing a method for generating a modified audio speech signal from an audio speech input signal, when being executed on a computer or signal processor, wherein the audio speech input signal comprises a plurality of speech subband signals, wherein the modified audio speech signal comprises a plurality of modified subband signals, wherein the method comprises:
generating weighting information for each speech subband signal of the plurality of speech subband signals depending on a signal power of said speech subband signal,
modifying each speech subband signal of the plurality of speech subband signals by applying the weighting information of said speech subband signal on said speech subband signal to acquire a modified subband signal of the plurality of modified subband signals, and
outputting the modified audio speech signal,
wherein generating the weighting information for each of the plurality of speech subband signals and modifying each of the speech subband signals are conducted so that a first speech subband signal of the plurality of speech subband signals comprising a first signal power is amplified with a first degree, and so that a second speech subband signal of the plurality of speech subband signals comprising a second signal power is amplified with a second degree, wherein the first signal power is greater than the second signal power, and wherein the first degree is lower than the second degree.