US 10,856,094 B2Granted

Method and device for sound source localization

AssigneesNanjing Twirling Technology Co., Ltd., Beijing Twirling In Time Co., Ltd.

InventorsXuejing Sun, Xingtao Zhang, Chen Zhang

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method and an apparatus for locating a sound source are provided. The method includes: obtaining M channels of audio signals of a preset format by microphone arrays located in different planes (S100); preprocessing the M channels of audio signals of the preset format, and projecting them onto the same plane, so as to obtain N channels of audio signals, where M≥N (S200); performing a time-frequency transformation on each of the N channels of audio signals, so as to obtain frequency domain signals of the N channels of audio signals (S300); further calculating a covariance matrix of the frequency domain signals and performing a smoothing process (S400); performing an eigenvalue decomposition of the smoothed covariance matrix (S500); estimating the sound source direction according to an eigenvector corresponding to the maximum eigenvalue, so as to obtain a sound source orientation parameter (S600).

Claims8

Dependent claims

A method for locating a sound source comprising:

step 1:
obtaining M channels of audio signals of a preset format by using microphone arrays located on different planes, wherein M is a positive integer;
step 2:
preprocessing the M channels of audio signals of the preset format, and projecting the M channels of audio signals of the preset format onto a same plane to obtain N channels of audio signals, wherein N is a positive integer, and M≥N;
step 3:
performing a time-frequency transform on each of the N channels of audio signals to obtain frequency domain signals of the N channels of audio signals;
step 4:
calculating covariance matrices of the frequency domain signals, and performing a smoothing process on each of the covariance matrices to obtain smoothed covariance matrices;
step 5:
performing an eigenvalue decomposition on each of the smoothed covariance matrices to obtain N eigenvalues and corresponding eigenvectors;
and
step 6:
estimating a direction of the sound source according to an eigenvector corresponding to a maximum eigenvalue of the N eigenvalues, to obtain sound source orientation parameters,
wherein in the step 1, M=4, the preset format is an Ambisonic A format, and the four channels of audio signals (LFU, RFD, LBD, RBU) are located on different planes,
wherein a specific process of the preprocessing in the step 2 is:
converting he four channels of audio signals of the Ambisonic A format into three (N=3) channels of audio signals (L, R, S) in the same plane by a conversion matrix A:
[
L
R
S
]
=
A
×
[
LFU
RFD
LBD
RBU
]
wherein the conversion matrix
A
=
[
a
11
a
12
a
13
a
14
a
21
a
22
a
23
a
24
a
31
a
32
a
33
a
34
]
,
and values of elements a11, a12, . . . , a34 of the conversion matrix A are constants and are determined by different sound source scenes.

58 more elements

A device for locating a sound source, comprising:

an acquisition unit of an audio signal of a preset format, a signal preprocessing unit, a time-frequency transform unit, a frequency domain signal processing unit, and a sound source orientation estimation unit, wherein
the acquisition unit of the audio signal of the preset format is configured to obtain M channels of audio signals of the preset format by using microphone arrays located on different planes, and send the M channels of audio signals of the preset format to the signal preprocessing unit, wherein M is a positive integer and M=4;
the signal preprocessing unit is configured to preprocess the M channels of audio signals of the preset format and project the M channels of audio signals of the preset format onto a same plane to obtain N channels of audio signals, and send the N channels of audio signals to the time-frequency transform unit, wherein N is a positive integer, and M≥N;
the time-frequency transform unit is configured to perform a time-frequency transform on each of the N channels of audio signals to obtain frequency domain signals of the N channels of audio signals;
the frequency domain signal processing unit is configured to process the frequency domain signals, calculate covariance matrices of the frequency domain signals and perform a smoothing process, further perform an eigenvalue decomposition on the covariance matrices to obtain eigenvalues and eigenvectors, and send the eigenvalues and eigenvectors to the sound source orientation estimation unit;
and
the sound source orientation estimation unit is configured to estimate a direction of the sound source according to an eigenvector corresponding to a maximum eigenvalue of the eigenvalues, to obtain sound source orientation parameters,
wherein the preset format is an Ambisonic A format, and four channels of audio signals (LFU, RFD, LBD, RBU) are located on the different planes,
wherein the signal preprocessing unit is configured to
convert the four channels of audio signals of the Ambisonic A format into three (N=3) channels of audio signals (L, R, S) in the same plane by a conversion matrix A:
[
L
R
S
]
=
A
×
[
LFU
RFD
LBD
RBU
]
wherein the conversion matrix
A
=
[
a
11
a
12
a
13
a
14
a
21
a
22
a
23
a
24
a
31
a
32
a
33
a
34
]
,
and values of elements a11, a12, . . . , a34 of the conversion matrix A are constants and are determined by different sound source scenes.

52 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/515,036Filed 18 Jul 2019 · continuation in part
Priority: 22 Jan 2017
Published: US 2019/0342688 A17 Nov 2019
Granted: 1 Dec 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification13 codes

H04S 3/02
Systems employing more than two channels, e.g. quadraphonic: of the matrix type, i.e. in which input signals are combined algebraically, e.g.…
G01S 5/18
Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance…:…
G01S 5/30
Using ultrasonic, sonic or infrasonic waves: determining absolute distances from a plurality of spaced points of known location
H04R 3/005
Circuits for transducers: for combining the signals of two or more microphones