# US 12,120,490 B2 · Filter coefficient optimization apparatus, filter coefficient optimization method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,120,490 B2
- Document ID: US12120490B2
- Status: Granted
- Kind: B2
- Application number: 17/801,754
- Filed: 2020-02-28
- Published: 2023-03-23
- Granted: 2024-10-15
- Earliest claimed: 2020-02-28
- Assignees: Nippon Telegraph and Telephone Corporation
- Applicants: Nippon Telegraph and Telephone Corporation
- Inventors: Ryotaro Sato; Kenta Niwa
- Web page: https://patentgrep.com/patent/US12120490B2

## CPC

- H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)}
- G06F 17/11: for solving equations {, e.g. nonlinear equations, general mathematical optimization problems (optimization specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G10K 11/34: using electrical steering of transducer arrays, e.g. beam steering {(constructional aspects B06B1/0607, B06B1/085)}
- G10L 21/0208: Noise filtering
- H04R 3/04: for correcting frequency response
- G10L 2021/02166: {Microphone arrays; Beamforming}
- H04R 1/406: {microphones}
- H04R 2430/20: Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic (H04R2203/12 takes precedence)
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10K: SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Provided is a filter coefficient optimization technology that makes it possible to design a stable beamformer having a good quality by considering the relationship of a filter coefficient between adjacent frequency bins. A filter coefficient optimization apparatus includes an optimization unit that calculates an optimum value of a filter coefficient w={w1, . . . , wF} (wf is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (target sound) from D sound source, af,d being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from an angular direction θd in which a sound source d exists, the sound wave being a plane wave, the optimization unit calculating the optimum value based on an optimization problem of a cost function defined using a sum of a sum of a cost function LMV_f(wf) and a predetermined regularization term, under a predetermined constraint condition, the predetermined regularization term being defined using a difference in phase between adjacent frequency bins relevant to a response wfHaf,d of the beamformer in the frequency bin f for the angular direction θd.

## Claims

### Claim 1

A filter coefficient optimization apparatus including an optimization unit that calculates an optimum value w* of a filter coefficient w={w1, . . . , WF} (wf(f=1, . . . , F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D),
D being an integer equal to or more than 1,
the optimization unit calculating the optimum value w* based on an optimization problem minwL(w) relevant to the filter coefficient w, under a predetermined constraint condition,
L(w) being a cost function relevant to the filter coefficient w and being defined using a sum of a sum Σf=1FLMV_f(wf) of a cost function LMV_f(wf) and a predetermined regularization term,
LMV_f(wf)=wfHRfwf(f=1, . . . , F) being a cost function relevant to the filter coefficient wf.
Rf(f=1, . . . , F) being a spatial correlation matrix for sound other than the target sound relevant to the frequency bin f,
the predetermined regularization term being defined using a difference in phase between adjacent frequency bins relevant to a response wfHaf,d (f=1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for an angular direction θd.
θd(d=1, . . . , D) being the angular direction in which a sound source d exists, and
af,d(f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θd, the sound wave being a plane wave; and
an output unit that sets the optimum value w* to the frequency bin of the beamformer and emphasizes at least a part of the sound wave as the target sound.

### Claim 2 (depends on 1)

The filter coefficient optimization apparatus according to claim 1, wherein
the predetermined regularization term is ηΣf=1F−1Σd=1D|∠(wfHaf,d)−∠(wf+1Haf+1,d)|2πor ηΣf=1F−2Σd=1D|∠(wfHaf,d)−2∠(wf+1Haf+1Haf+1,d)+∠(wf+2Haf+2,d)|2π.
η being a predetermined positive number, and
∠(wfHaf,d) (f=1, . . . , F, d=1, . . . , D) being the phase of the response wfHaf,d of the beamformer in the frequency bin f for the angular direction θd.

### Claim 3 (depends on 1)

The filter coefficient optimization apparatus according to claim 1, wherein:
the predetermined regularization term is
ηΣf=1F−1Σd=1D|cf,d-cf+1,d|c.
η being a predetermined positive number,
C being an integer equal to or more than 1,
∠(wfHaf,d) (f=1, . . . , F, d=1, . . . , D) being the phase of the response wfHaf,d of the beamformer in the frequency bin f for the angular direction θd, and
cf,d(f=1, . . . , F, d=1, . . . , D) being a discrete variable having one value of 1, . . . , C that satisfies ∠(wfHaf,d)∈[2π(cf,d−1)/C, 2πcf,d/C] for the phase ∠(wfHaf,d).

### Claim 4 (depends on 3)

The filter coefficient optimization apparatus according to claim 3, wherein the predetermined constraint condition is expressed by the following expression:
wfHαf,d=1 [Math. 32]
(f=1, . . . , F).

### Claim 5 (depends on 3)

The filter coefficient optimization apparatus according to claim 3, wherein the predetermined constraint condition is expressed by the following expression:
wfHαf,d≥1 [Math. 33]
(f=1, . . . , F, d=1, . . . , D).

### Claim 6 (depends on 4)

The filter coefficient optimization apparatus according to claim 4, wherein
the optimization unit includes
a candidate calculation unit that calculates αf[cf] for all values that a discrete variable cf can have, for each frequency bin f, by the following expression, and adopts copt=argmincαF[c] as a value of a variable copt,
cf=(cf,1, . . . , cf,D) (f=1, . . . , F) being the discrete variable that is defined by a discrete variable cf,1, . . . , cf,D.
[
Math
.
34
]

w
f
dp
[
c
f
]
←
argmin
w
f
⁢
L
MV
f
(
w
f
)
s
.
t
.
∠
⁡
(
w
f
H
⁢
a
f
,
d
)
∈
[
2
⁢
π
⁡
(
c
f
,
d
-
1
)
/
C
,
2
⁢
π
⁢
c
f
,
d
/
C
]
(
d
=
1
,
…
,
D
)
,
w
f
H
⁢
a
f
,
1
=
1
c
f
prev
[
c
f
]
←
arg
⁢
min
c
f
-
1
′
⁢
(
α
f
-
1
[
c
f
-
1
′
]
+
L
^
η
⁢
f
-
1
(
c
f
-
1
′
,
c
f
)
)
α
f
[
c
f
]
←
α
f
-
1
[
c
f
prev
[
c
f
]
]
+
L
^
η
⁢
f
-
1
(
c
f
prev
[
c
f
]
,
c
f
)
+
L
MV
f
(
w
f
dp
[
c
f
]
)
(
⋀
L
η
⁢
f
(
c
f
,
c
f
+
1
)
=
η
⁢
∑
d
=
1
D
❘
"\[LeftBracketingBar]"
c
f
,
d
-
c
f
+
1
,
d
❘
"\[RightBracketingBar]"
c
)
,
and
an optimum value determination unit that calculates an optimum value wf* of the filter coefficient wf and the value of the variable copt for the frequency bin f, in descending order from F to 1, by the following expression, and obtains the optimum value w* from w*={w1*, . . . , wF*}:
wf*←wfdp[copt]
copt←cfprev[copt]. [Math. 35]

### Claim 7 (depends on 5)

The filter coefficient optimization apparatus according to claim 5, wherein:
Λ
(
f
,
d
)
,
c
f
,
d
(
γ
f
,
d
)
=
{
0
⁢
(
R
⁡
(
γ
f
,
d
⁢
e
-
2
⁢
π
⁢
j
⁡
(
c
f
,
d
+
1
)
/
2
⁢
C
)
≥
1
,
2
⁢
π
⁢
c
f
,
d
C
≤
∠γ
f
,
d
≤
2
⁢
π
⁡
(
c
f
,
d
+
1
)
C
)
∞
⁡
(
otherwise
)
the optimization unit calculates the optimum value w* by solving an optimization problem min{c_f,w_f}(Σf=1FLMV_f(wf)+Σf=1FΣd=1D{circumflex over ( )}(f,d)c_f,d(wfHaf,d)+ηΣf=1F−1Σd=1D|df,d-cf+1,3|c) relevant to the filter coefficient w and a discrete variable c1, . . . , cF, instead of solving the optimization problem minwL(w),
cf=(cf,1, . . . , cf,D) (f=1. . . , F) being the discrete variable that is defined by the discrete variable cf,1, . . . , cf,D, and
{circumflex over ( )}(f,d),c_f,d(f=1, . . . , F, d=1, . . . , D) is a function relevant to a variable γf,d that is defined by the following expression (γf,d=wfHaf,d):
[
Math
.
36
]

Λ
(
f
,
d
)
,
c
f
,
d
(
γ
f
,
d
)
=
{
0
⁢
(
R
⁡
(
γ
f
,
d
⁢
e
-
2
⁢
π
⁢
j
⁡
(
c
f
,
d
+
1
)
/
2
⁢
C
)
≥
1
,
2
⁢
π
⁢
c
f
,
d
C
≤
∠γ
f
,
d
≤
2
⁢
π
⁡
(
c
f
,
d
+
1
)
C
)
∞
⁢
(
otherwise
)
.

### Claim 8 (depends on 7)

The filter coefficient optimization apparatus according to claim 7, wherein
the optimization unit includes
a candidate calculation unit that calculates αf[cf] for all values that the discrete variable f can have, for each frequency bin f, by the following expression, and adopts as a Copt=argmincαF[c] value of a variable copt:
[
Math
.
37
]
w
f
dp
[
c
f
]
←
arg
⁢
min
w
f
⁢
(
L
MV
f
(
w
f
)
+
∑
d
=
1
D
Λ
(
f
,
d
)
,
c
f
,
d
(
w
f
H
⁢
a
f
,
d
)
)
c
f
prev
[
c
f
]
←
argmin
c
f
-
1
′
(
α
f
-
1
[
c
f
-
1
′
]
+
L
^
η
⁢
f
-
1
(
c
f
-
1
′
,
c
f
)
)
α
f
[
c
f
]
←
α
f
-
1
[
c
f
prev
[
c
f
]
]
+
L
^
η
⁢
f
-
1
(
c
f
prev
[
c
f
]
,
c
f
)
+
L
MV
f
(
w
f
dp
[
c
f
]
)
+
∑
d
=
1
D
Λ
(
f
,
d
)
,
c
f
,
d
(
w
f
dp
[
c
f
]
H
⁢
a
f
,
d
)
(
⋀
L
η
⁢
f
(
c
f
,
c
f
+
1
)
=
η
⁢
∑
d
=
1
D
❘
"\[LeftBracketingBar]"
c
f
,
d
-
c
f
+
1
,
d
❘
"\[RightBracketingBar]"
c
)
⁢
and
an optimum value determination unit that calculates an optimum value wf* of the filter coefficient wf and the value of the variable Copt for the frequency bin f, in descending order from F to 1, by the following expression, and obtains the optimum value w* from w*={w1*, . . . ,wF*}:
wf*←wfdp[copt]
copt←cfprev[copt]. [Math. 38]

### Claim 9

A filter coefficient optimization apparatus including an optimization unit that calculates an optimum value w* of a filter coefficient w={w1, . . . , wF} (wf(f=1, . . . , F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D),
D being an integer equal to or more than 1,
the optimization unit calculating the optimum value w* by performing derivation so as to reduce a difference in phase between adjacent frequency bins relevant to a response wfHaf,d(f=1, . . . , , F, d=1, . . . , D) of the beamformer in the frequency bin f for an angular direction θd,
θd(d=1, . . . , D) being the angular direction in which a sound source d exists,
af,d(f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θd, the sound wave being a plane wave; and
an output unit that sets the optimum value w* to the frequency bin of the beamformer and emphasizes at least a part of the sound wave as the target sound.

### Claim 10

A filter coefficient optimization method including an optimization step in which a filter coefficient optimization apparatus calculates an optimum value w* of a filter coefficient w={w1, . . . , wF} (wf(f=1, F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D),
D being an integer equal to or more than 1,
the optimization step being a step of calculating the optimum value w* based on an optimization problem minwL(w) relevant to the filter coefficient w, under a predetermined constraint condition,
L(w) being a cost function relevant to the filter coefficient w and being defined using a sum of a sum Σf=1FlMV_f(wf) of a cost function LMV_f(wf) and a predetermined regularization term,
LMV_f(wf)=wfHRfwf(f=1, . . . , F) being a cost function relevant to the filter coefficient wf,
Rf(f=1, . . . , F) being a spatial correlation matrix for sound other than the target sound relevant to the frequency bin f,
the predetermined regularization term being defined using a difference in phase between adjacent frequency bins relevant to a response wfHaf,d(f=1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for an angular direction θd,
θd(d=1, . . . , D) being the angular direction in which a sound source d exists, and
af,d(f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θd, the sound wave being a plane wave; and
an output step that sets the optimum value w* to the frequency bin of the beamformer and emphasizes at least a part of the sound wave as the target sound.

### Claim 11

A filter coefficient optimization method including an optimization step in which a filter coefficient optimization apparatus calculates an optimum value w* of a filter coefficient w={w1, . . . , wF} (wf(f=1, . . . , F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D),
D being an integer equal to or more than 1,
the optimization step being a step of calculating the optimum value w* by performing derivation so as to reduce a difference in phase between adjacent frequency bins relevant to a response wfHaf,d(f=1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for an angular direction θd,
θd(d=1, . . . ,D) being the angular direction in which a sound source d exists, and
af,d(f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θd, the sound wave being a plane wave; and
an output step that sets the optimum value w* to the frequency bin of the beamformer and emphasizes at least a part of the sound wave as the target sound.

### Claim 12 (depends on 1)

A non-transitory computer-readable recording medium storing a program that causes a computer to function as the filter coefficient optimization apparatus according to claim 1.

### Claim 13 (depends on 9)

A non-transitory computer-readable recording medium storing a program that causes a computer to function as the filter coefficient optimization apparatus according to claim 9.