Multi-modal audio processing for voice-controlled devices
AssigneeSoundHound, Inc.
InventorKarl Stahl
Life of this patent
Estimating the term…
Abstract
A voice-controlled device includes a microphone to receive a set of sound waves that includes speech uttered by a user and other sound, and to output a first audio signal that includes a contribution from the speech uttered by the user and a contribution from the other sound. The device also includes a receiver to receive an electromagnetic signal and to output a second audio signal obtained from the electromagnetic signal. An audio pre-processor of the device processes the first audio signal using the second audio signal to reduce the contribution from the other sound in a processed audio signal. The voice-controlled device then provides the processed audio signal to a speech recognition module to determine a voice command issued by the user.
Claims36
A method of processing an audio signal for a voice-controlled device, the method comprising:
- receiving a set of sound waves at a microphone of the voice-controlled device, the set of sound waves comprising speech uttered by a user and other sound;
- converting, using the microphone, the set of sound waves into a first audio signal that includes a contribution from the speech uttered by the user and a contribution from the other sound;
- receiving, at a receiver of the voice-controlled device, an electromagnetic signal;
- obtaining a second audio signal from the electromagnetic signal;
- processing the first audio signal using the second audio signal to reduce the contribution from the other sound in a processed audio signal;
- and
- performing speech recognition on the processed audio signal to determine a voice command issued by the user.
A voice-controlled device comprising:
- a microphone configured to receive a set of sound waves comprising speech uttered by a user and other sound, and to output a first audio signal that includes a contribution from the speech uttered by the user and a contribution from the other sound;
- a receiver configured to receive an electromagnetic signal and to output a second audio signal obtained from the electromagnetic signal;
- and
- an audio pre-processor configured to process the first audio signal using the second audio signal to reduce the contribution from the other sound in a processed audio signal;
- wherein the voice-controlled device is configured to provide the processed audio signal to a speech recognition module to determine a voice command issued by the user.
A non-transitory computer-readable storage medium storing instructions which, when executed by at least one processor, program the at least one processor to:
- obtain a first audio signal that includes a contribution from speech uttered by a user and a contribution from other sound, the first audio signal derived from a set of sound waves received at a microphone of a voice-controlled device, the set of sound waves comprising the speech uttered by the user and the other sound;
- obtain a second audio signal, the second audio signal derived from an electromagnetic signal received at a receiver of the voice-controlled device;
- correlate the first audio signal and the second audio signal to generate one or more correlation parameters, the one or more correlation parameters indicating a time delay and/or a scaling factor for the second audio signal;
- reduce the contribution from the other sound in the first audio signal using the one or more correlation parameters to generate a processed audio signal;
- and
- provide the processed audio signal to a speech recognition module to determine a voice command issued by the user.
Dependencies are inferred from the claim text.