Multi-modal audio processing for voice-controlled devices
AssigneeSoundHound, Inc.
InventorKarl Stahl
Life of this patent
Estimating the term…
Abstract
A voice-controlled device includes a microphone to receive a set of sound waves that includes speech uttered by a user and other sound, and to output a first audio signal that includes a contribution from the speech uttered by the user and a contribution from the other sound. The device also includes a receiver to receive an electromagnetic signal and to output a second audio signal obtained from the electromagnetic signal. An audio pre-processor of the device processes the first audio signal using the second audio signal to reduce the contribution from the other sound in a processed audio signal. The voice-controlled device then provides the processed audio signal to a speech recognition module to determine a voice command issued by the user.
Claims32
A method of processing an audio signal for a voice-controlled device, the method comprising:
- receiving a set of sound waves at a microphone of the voice-controlled device, the set of sound waves comprising speech uttered by a user and other sound;
- converting, using the microphone, the set of sound waves into a first audio signal that includes a contribution from the speech uttered by the user and a contribution from the other sound;
- receiving, at a receiver of the voice-controlled device, an electromagnetic signal carrying a second audio signal;
- extracting the second audio signal from the electromagnetic signal;
- correlating the first audio signal with the second audio signal to calculate a correlation value;
- in response to the correlation value being larger than a threshold, processing the first audio signal using the second audio signal to reduce the contribution from the other sound in a processed audio signal;
- and
- performing speech recognition on the processed audio signal to determine a voice command issued by the user.
4-5. (canceled)
(canceled)
(canceled)
15-16. (canceled)
(canceled)
A voice-controlled device comprising:
- a microphone to receive a set of sound waves comprising speech uttered by a user and other sound, and to output a first audio signal that includes a contribution from the speech uttered by the user and a contribution from the other sound;
- a receiver to receive an electromagnetic signal and to output a second audio signal obtained from the electromagnetic signal;
- a correlator to generate one or more correlation parameters from a correlation of the first audio signal and the second audio signal;
- and
- an audio pre-processor to process the first audio signal using the second audio signal and at least one of the one or more correlation parameters to reduce the contribution from the other sound in a processed audio signal;
- wherein the voice-controlled device is configured to provide the processed audio signal to a speech recognition module to determine a voice command issued by the user.
(canceled)
(canceled)
(canceled)
28-30. (canceled)
A non-transitory computer-readable storage medium storing instructions which, when executed by at least one processor, program the at least one processor to:
- obtain a first audio signal that includes a contribution from speech uttered by a user and a contribution from other sound, the first audio signal derived from a set of sound waves received at a microphone of a voice-controlled device, the set of sound waves comprising the speech uttered by the user and the other sound;
- obtain a second audio signal, the second audio signal derived from an electromagnetic signal received at a receiver of the voice-controlled device;
- correlate the first audio signal and the second audio signal to generate one or more correlation parameters, the one or more correlation parameters indicating a time delay and/or a scaling factor for the second audio signal;
- in response to the scaling factor being larger than a threshold, reduce the contribution from the other sound in the first audio signal using the second audio signal and the one or more correlation parameters to generate a processed audio signal;
- and
- provide the processed audio signal to a speech recognition module to determine a voice command issued by the user.
(canceled)
Dependencies are inferred from the claim text.