Back to results
US 11,107,492 B1Granted
Omni-directional speech separation
AssigneeAmazon Technologies, Inc.
InventorWai Chung Chu
Life of this patent
Estimating the term…
Abstract
A system configured to perform directional speech separation using three or more microphones. The system may dynamically associate direction-of-arrivals with one or more audio sources in order to generate output audio data that separates each of the audio sources. Using three or more microphones, the system may separate audio sources covering 360 degrees surrounding the microphone array, whereas a two-microphone implementation is limited to 180 degrees. The system identifies a target direction for each audio source, dynamically determines directions that are correlated with the target direction, and generates output signals for each audio source. The system may associate individual frequency bands with specific directions based on a phase difference detected by two or more microphones.
Claims20
A computer-implemented method, comprising:
- detecting, by a first microphone of a microphone array, first audio within a first audible frequency range;
- detecting, by a second microphone of the microphone array, second audio within the first audible frequency range;
- determining first audio data using the first microphone, a portion of the first audio data representing the first audio;
- determining second audio data using the second microphone, a first portion of the second audio data representing the second audio;
- determining, using a third microphone of the microphone array, third audio data, a first portion of the third audio data representing third audio within the first audible frequency range;
- determining a first lag estimate value indicating a first phase difference between the portion of the first audio data and the first portion of the second audio data;
- determining that the first lag estimate value is within a first range of values, the first range of values corresponding to one of a first direction or a second direction of a plurality of directions;
- determining a first amount of energy of the portion of the first audio data;
- determining that the first amount of energy exceeds a threshold value, indicating that the portion of the first audio data corresponds to an audio source;
- determining, using the second audio data and the third audio data, a second portion of the second audio data that corresponds to the first direction;
- determining a second amount of energy of the second portion of the second audio data;
- determining, using the second audio data and the third audio data, a third portion of the second audio data that corresponds to the second direction;
- determining a third amount of energy of the second portion of the second audio data;
- determining that the second amount of energy exceeds the third amount of energy;
- determining, in response to the second amount of energy exceeding the third amount of energy, that the audio source corresponds to the first direction;
- determining that the second amount of energy exceeds the first amount of energy;
- in response to the second amount of energy exceeding the first amount of energy, associating the audio source with a first microphone pair comprising the first microphone and the second microphone;
- and
- generating association data indicating that two or more of the plurality of directions are associated with the audio source and the first microphone pair, the two or more of the plurality of directions including the first direction.
16 more elements
A computer-implemented method, comprising:
- receiving first audio data associated with a first microphone;
- receiving second audio data associated with a second microphone;
- receiving third audio data associated with a third microphone;
- determining, based at least in part on a phase difference between a portion of the first audio data and a portion of the second audio data, that a first energy value associated with an audio source corresponds to one of a first direction or a second direction of a plurality of directions;
- using the second audio data and the third audio data to determine a second energy value corresponding to the first direction;
- using the second audio data and the third audio data to determine a third energy value corresponding to the second direction;
- using the second energy value and the third energy value to determine that the audio source corresponds to the first direction;
- determining that the first energy value is less than the second energy value;
- and
- associating the audio source with the first direction and a first microphone pair comprising the first microphone and the second microphone.
7 more elements
A system comprising:
- at least one processor;
- and
- memory including instructions operable to be executed by the at least one processor to cause the system to:
- receive first audio data associated with a first microphone;
- receive second audio data associated with a second microphone;
- receive third audio data associated with a third microphone;
- determine, based at least in part on a phase difference between a portion of the first audio data and a portion of the second audio data, that a first energy value associated with an audio source corresponds to one of a first direction or a second direction of a plurality of directions;
- use the second audio data and the third audio data to determine a second energy value corresponding to the first direction;
- use the second audio data and the third audio data to determine a third energy value corresponding to the second direction;
- use the second energy value and the third energy value to determine that the audio source corresponds to the first direction;
- determine that the first energy value is less than the second energy value;
- and
- associate the audio source with the first direction and a first microphone pair comprising the first microphone and the second microphone.
10 more elements
Dependencies are inferred from the claim text.