# US 10,455,325 B2 · Direction of arrival estimation for multiple audio content streams

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,455,325 B2
- Document ID: US10455325B2
- Status: Granted
- Kind: B2
- Application number: 16/234,509
- Filed: 2018-12-27
- Published: 2019-07-04
- Granted: 2019-10-22
- Earliest claimed: 2017-12-28
- Assignees: Knowles Electronics, LLC
- Applicants: Knowles Electronics, LLC
- Inventors: John Woodruff; Pratik Shah; Sai Ravi Teja Pulugurtha; Harsha Rao
- Web page: https://patentgrep.com/patent/US10455325B2

## CPC

- H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)}
- G01S 3/80: using ultrasonic, sonic or infrasonic waves
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G10L 21/0272: Voice signal separating
- G10L 21/028: using properties of sound source
- G10L 25/30: using neural networks
- H04R 1/406: {microphones}
- H04R 3/04: for correcting frequency response
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method for estimating directions of arrival of audio signals based on categories is disclosed herein. The method includes receiving an audio signal; generating a plurality of filters based on the audio signal, each of the filters corresponding to one of a plurality of sound content categories; separating the audio signal into a plurality of content-based audio signals by applying the filters to the audio signal, each of the content-based audio signals contains a content of a corresponding sound content category among the plurality of sound content categories; for each of the content-based audio signals, generating a direction of arrival (DOA) for a sound source of the content-based audio signal.

## Claims

### Claim 1

A method for estimating directions of arrival based on categories, comprising:
receiving an audio signal;
providing the audio signal to a neural network that has been trained using known sound content corresponding to the plurality of sound content categories;
generating, by the neural network, a plurality of filters based on the audio signal, each of the filters corresponding to one of the plurality of sound content categories; and
separating the audio signal into a plurality of category specific audio signals using the filters, the category specific audio signals respectively corresponding to the plurality of sound content categories; and
for each of the category specific audio signals, generating a direction of arrival (DOA) for a sound source of the category specific audio signal.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the DOA further includes, for each of the category specific audio signals, generating a probability value indicating a likelihood of the DOA of the sound source.

### Claim 3 (depends on 2)

The method of claim 2, further comprising extracting a plurality of features from the audio signal, wherein generating the DOA for the sound source and the probability value is further based on the extracted features.

### Claim 4 (depends on 1)

The method of claim 1, wherein generating the DOA comprises:
extracting a plurality of features from the audio signal;
generating one or more estimated DOAs for one or more sound source directions and probabilities associated with the estimated DOAs based on the extracted features; and
selecting the DOA from the one or more estimated DOAs by comparing the probabilities associated with the estimated DOAs.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
tracking the DOA of the sound source of the category specific audio signal across multiple time frames.

### Claim 6 (depends on 5)

The method of claim 5, wherein the DOA of the sound source of the category specific audio signal is tracked using a Kalman filter.

### Claim 7 (depends on 1)

The method of claim 1, wherein the method further comprises generating a plurality of features from the audio signal, and wherein generating the plurality of filters is further based on the generated features.

### Claim 8 (depends on 1)

The method of claim 1, further comprising converting the audio signal from a time domain to a frequency domain, wherein the plurality of filters comprise frequency domain filters, and wherein the plurality of filters are applied to the frequency domain representation of the audio signal.

### Claim 9 (depends on 8)

The method of claim 8, further comprising converting the plurality of category specific audio signals from the frequency domain into the time domain.

### Claim 10 (depends on 1)

The method of claim 1, wherein the neural network has been trained by:
combining at least a first training signal of a first known sound content category and a second training signal of a second known sound content category into a combined training audio signal; and
training the neural network by feeding the combined training audio signal into the neural network and optimizing parameters of the neural network.

### Claim 11 (depends on 10)

The method of claim 10, wherein optimizing parameters of the neural network includes comparing a frequency mask generated by the neural network to an optimal frequency mask associated with one of the first and second audio signals.

### Claim 12 (depends on 10)

The method of claim 10, wherein the first and second training signals are clean signals having sound content corresponding the first and second known sound content categories, respectively.

### Claim 13 (depends on 1)

The method of claim 1, wherein each of the filters is a time-varying real-valued function of frequency, and a value of the time-varying real-valued function for a corresponding frequency and a corresponding time frame represents a level of signal attenuation for the corresponding frequency at the corresponding time frame.

### Claim 14 (depends on 13)

The method of claim 13, wherein the separating the audio signal into a plurality of category specific audio signals by applying the filters to the audio signal comprises:
separating the audio signal into a plurality of category-specific audio signals by multiplying the audio signal by the time-varying real-valued functions.

### Claim 15 (depends on 1)

The method of claim 1, further comprising:
reproducing a virtual reality sound stage using the category specific audio signals and the DOAs of the sound sources.

### Claim 16

A method of reproducing a virtual sound environment, comprising:
receiving an audio signal;
generating a plurality of filters based on the audio signal, each of the filters corresponding to one of a plurality of sound content categories, the sound content categories including a target sound content category and one or more ambient sound content categories; and
separating the audio signal into a plurality of category specific audio signals using the filters, the category specific audio signals includes a target category specific audio signal of the target sound content category and one or more ambient category specific audio signals of the ambient sound content categories;
for each of the category specific audio signals, generating a direction of arrival (DOA) for a sound source of the category specific audio signal and a probability value indicating a likelihood of the DOA of the sound source; and
reproducing a virtual sound environment by replaying the category specific audio signals using a plurality of speakers, wherein for each of the category specific audio signals, designation of one or more of the speakers that replay the category specific audio signal and determination of replay parameters are based on the DOA of the sound source of the category specific audio signal.

### Claim 17 (depends on 16)

The method of claim 16, wherein generating the plurality of filters is performed using a neural network that has been trained using known sound content corresponding to the plurality of sound content categories.

### Claim 18 (depends on 17)

The method of claim 17, wherein the neural network has been trained by:
combining a first training signal of the target sound content category and a second training signal of one of the ambient sound content categories into a combined training audio signal; and
training the neural network by feeding the combined training audio signal into the neural network and optimizing parameters of the neural network.

### Claim 19 (depends on 18)

The method of claim 18, wherein optimizing parameters of the neural network includes comparing a frequency mask generated by the neural network to an optimal frequency mask associated with one of the first and second training signals.

### Claim 20 (depends on 18)

The method of claim 18, wherein the first and second training signals are clean signals having sound content corresponding to the first and second known sound content categories, respectively.