Back to results
US 9,813,810 B1Granted
Multi-microphone neural network for sound recognition
AssigneeGoogle Inc.
InventorRajeev Conrad Nongpiur
Life of this patent
Estimating the term…
Abstract
A neural network is provided for recognition and enhancement of multi-channel sound signals received by multiple microphones, which need not be aligned in a linear array in a given environment. Directions and distances of sound sources may also be detected by the neural network without the need for a beamformer connected to the microphones. The neural network may be trained by knowledge gained from free-field array impulse responses obtained in an anechoic chamber, array impulse responses that model simulated environments of different reverberation times, and array impulse responses obtained in actual environments.
Claims23
A method comprising:
- generating a plurality of free-field array impulse responses representing impulse responses of a plurality of microphones in free-field conditions over a plurality of directions based on sound events received by the microphones;
- generating a first plurality of array impulse responses that model a plurality of simulated environments of different reverberation times, wherein the simulated environments differ from the free-field conditions in which the free-field array impulse responses are generated;
- generating a second plurality of array impulse responses in one or more actual environments, wherein the one or more actual environments differ from the free-field conditions in which the free-field array impulse responses are generated;
- and
- training a neural network for sound recognition based on one or more of the free-field array impulse responses, the first plurality of array impulse responses, and the second plurality of array impulse responses, wherein training the neural network for sound recognition includes:
- applying positive training labels to input sound signals corresponding to sound events from one or more prescribed directions;
- and
- applying negative training labels are applied to input sound signals corresponding to sound events from directions other than the one or more prescribed directions.
5 more elements
An apparatus comprising:
- a memory;
- and
- a processor communicably coupled to the memory, the processor configured to execute instructions to:
- generate a plurality of free-field array impulse responses representing impulse responses of a plurality of microphones in free-field conditions over a plurality of directions based on sound events received by the microphones;
- generate a first plurality of array impulse responses that model a plurality of simulated environments of different reverberation times, wherein the simulated environments differ from the free-field conditions in which the free-field array impulse responses are generated;
- generate a second plurality of array impulse responses in one or more actual environments, wherein the one or more actual environments differ from the free-field conditions in which the free-field array impulse responses are generated;
- and
- train a neural network for sound recognition based on one or more of the free-field array impulse responses, the first plurality of array impulse responses, and the second plurality of array impulse responses, wherein the instructions to train the neural network for sound recognition include instructions to:
- apply positive training labels to input sound signals corresponding to sound events from one or more prescribed directions;
- and
- apply negative training labels are applied to input sound signals corresponding to sound events from directions other than the one or more prescribed directions.
8 more elements
A sound system comprising:
- a plurality of microphones;
- and
- a neural network comprising a plurality of layers, wherein one or more of the layers of the neural network is configured to be trained for sound recognition by:
- generating a plurality of free-field array impulse responses representing impulse responses of a plurality of microphones in free-field conditions over a plurality of directions based on sound events received by the microphones;
- generating a first plurality of array impulse responses that model a plurality of simulated environments of different reverberation times, wherein the simulated environments differ from the free-field conditions in which the free-field array impulse responses are generated;
- generating a second plurality of array impulse responses in one or more actual environments, wherein the one or more actual environments differ from the free-field conditions in which the free-field array impulse responses are generated;
- and
- training a neural network for sound recognition based on one or more of the free-field array impulse responses, the first plurality of array impulse responses, and the second plurality of array impulse responses, wherein training the neural network for sound recognition includes:
- applying positive training labels to input sound signals corresponding to sound events from one or more prescribed directions;
- and
- applying negative training labels are applied to input sound signals corresponding to sound events from directions other than the one or more prescribed directions.
8 more elements
Dependencies are inferred from the claim text.