Multi-modal audio processing
AssigneeSoundHound AI IP, LLC
InventorKarl Stahl
Life of this patent
Estimating the term…
Abstract
A method for processing an audio signal involves receiving sound waves at a microphone, converting them into a first audio signal, and extracting a second audio signal from an electromagnetic signal received at a receiver. The first audio signal is correlated with the second audio signal to calculate a correlation value. If the correlation value exceeds a threshold, the first audio signal is processed using the second audio signal to reduce unwanted sound contributions, resulting in a processed audio signal. Further processing is then performed on the processed audio signal to determine a characteristic of the desired sound.
Claims20
A method of processing an audio signal, the method comprising:
- receiving a set of sound waves at a microphone of a device, the set of sound waves comprising a sound of interest and other sound;
- converting, using the microphone, the set of sound waves into a first audio signal that includes a contribution from the sound of interest and a contribution from the other sound;
- receiving, at a receiver of the device, an electromagnetic signal carrying a second audio signal;
- extracting the second audio signal from the electromagnetic signal;
- correlating the first audio signal with the second audio signal to calculate a correlation value;
- in response to the correlation value being larger than a threshold, processing the first audio signal using the second audio signal to reduce the contribution from the other sound in a processed audio signal;
- and
- performing further processing on the processed audio signal to determine a characteristic of the sound of interest.
A device comprising:
- a microphone to receive a set of sound waves comprising sound of interest and other sound, and to output a first audio signal that includes a contribution from the sound of interest and a contribution from the other sound;
- a receiver to receive an electromagnetic signal and to output a second audio signal obtained from the electromagnetic signal;
- a correlator to generate one or more correlation parameters from a correlation of the first audio signal and the second audio signal;
- and
- an audio pre-processor to process the first audio signal using the second audio signal and at least one of the one or more correlation parameters to reduce the contribution from the other sound in a processed audio signal;
- wherein the device is configured to provide the processed audio signal to at least one processor to determine a characteristic of the sound of interest.
A non-transitory computer-readable storage medium storing instructions which, when executed by at least one processor, program the at least one processor to:
- obtain a first audio signal that includes a contribution from sound of interest and a contribution from other sound, the first audio signal derived from a set of sound waves received at a microphone of a device, the set of sound waves comprising the sound of interest and the other sound;
- obtain a second audio signal, the second audio signal derived from an electromagnetic signal received at a receiver of the device;
- correlate the first audio signal and the second audio signal to generate one or more correlation parameters, the one or more correlation parameters indicating a time delay and/or a scaling factor for the second audio signal;
- in response to the scaling factor being larger than a threshold, reduce the contribution from the other sound in the first audio signal using the second audio signal and the one or more correlation parameters to generate a processed audio signal;
- and
- provide the processed audio signal to an audio processor to determine a characteristic of the sound of interest.
Dependencies are inferred from the claim text.