US 11,386,911 B1Granted

Dereverberation and noise reduction

AssigneeAmazon Technologies, Inc.

InventorsKanthasamy Chelliah, Wai Chung Chu, Andreas Schwarz, Berkant Tacer, Carlo Murgia

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system configured to improve audio processing by performing dereverberation and noise reduction during a communication session. The system may apply a two-channel dereverberation algorithm by calculating coherence-to-diffuse ratio (CDR) values and calculating dereverberation (DER) gain values based on the CDR values. While the DER gain values may be calculated at a first stage within the pipeline, the device may apply the DER gain values at a second stage within the pipeline. For example, the device may calculate the DER gain values prior to performing residual echo suppression (RES) processing but may apply the DER gain values after performing RES processing, in order to avoid excessive attenuation of the local speech. In addition to removing reverberation, the DER gain values also remove diffuse noise components, reducing an amount of noise reduction required. Thus, the device may soften noise reduction when the DER gain values are applied.

Claims20

Dependent claims

A computer-implemented method, the method comprising:

sending, by a device, reference audio data to a loudspeaker of the device to generate audio;
receiving first microphone audio data from a first microphone of the device, the first microphone audio data including a first representation of speech;
receiving second microphone audio data from a second microphone of the device, the second microphone audio data including a second representation of the speech;
performing, using the reference audio data and the first microphone audio data, echo cancellation to generate third microphone audio data corresponding to the first microphone;
performing, using the reference audio data and the second microphone audio data, echo cancellation to generate fourth microphone audio data corresponding to the second microphone;
determining a first signal-to-noise ratio (SNR) value for a first portion of the third microphone audio data, the first portion of the third microphone audio data representing a first audio frame;
determining that the first SNR value exceeds a threshold value indicating that noisy conditions are not present;
determining, using the first portion of the third microphone audio data and a first portion of the fourth microphone audio data, first coherence-to-diffuse ratio (CDR) data corresponding to the first audio frame;
determining, using the first CDR data, first gain values configured to suppress reverberations represented in the first portion of the third microphone audio data;
performing residual echo suppression on the first portion of the third microphone audio data to generate a first portion of first audio data;
performing dereverberation by applying the first gain values to the first portion of the first audio data to generate a first portion of second audio data;
determining, using the first portion of the second audio data, first noise estimate data;
and
performing noise reduction, using the first noise estimate data, on the first portion of the second audio data to generate a first portion of output audio data.

11 more elements

A computer-implemented method, the method comprising:

receiving reference audio data corresponding to audio generated by a loudspeaker;
receiving first microphone audio data associated with a first microphone;
receiving second microphone audio data associated with a second microphone;
performing, using the reference audio data and the first microphone audio data, echo cancellation to generate third microphone audio data associated with the first microphone;
performing, using the reference audio data and the second microphone audio data, echo cancellation to generate fourth microphone audio data associated with the second microphone;
determining, using the third microphone audio data and the fourth microphone audio data, first coherence-to-diffuse ratio (CDR) data;
determining, using the first CDR data, first gain values;
performing residual echo suppression on the third microphone audio data to generate first audio data;
and
performing, using the first gain values, dereverberation on the first audio data to generate second audio data.

7 more elements

A system comprising:

at least one processor;
and
memory including instructions operable to be executed by the at least one processor to cause the system to:
receive reference audio data corresponding to audio generated by a loudspeaker;
receive first microphone audio data associated with a first microphone;
receive second microphone audio data associated with a second microphone;
perform, using the reference audio data and the first microphone audio data, echo cancellation to generate third microphone audio data associated with the first microphone;
perform, using the reference audio data and the second microphone audio data, echo cancellation to generate fourth microphone audio data associated with the second microphone;
determine, using the third microphone audio data and the fourth microphone audio data, first coherence-to-diffuse ratio (CDR) data;
determine, using the first CDR data, first gain values;
perform residual echo suppression on the third microphone audio data to generate first audio data;
and
perform, using the first gain values, dereverberation on the first audio data to generate second audio data.

10 more elements

Dependencies are inferred from the claim text.