Adaptive enhancement of coded speech
AssigneeAmazon Technologies, Inc.
InventorsJan Buethe, Jean-Marc Valin, Ahmed Mustafa
Life of this patent
Estimating the term…
Abstract
Adaptive code enhancement includes receiving feature data of a speech waveform having a plurality of frames, inferring a bitrate based on a quantity of bits of the plurality of frames received from the decoder codec, generating one or more latent feature vectors in accordance with the bitrate and based on the feature data, receiving one or more pitch lags and a speech signal, calculating filter coefficients on a per-frame basis based on the one or more latent feature vectors and the one or more pitch lags, and modifying the speech signal using the one or more filter coefficients to generate a modified speech signal.
Claims18
A system, comprising:
- at least one processor;
- and
- a memory, storing program instructions that when executed by the at least one processor, cause the at least one processor to implement a deep neural network (DNN) machine learning model trained to enhance speech, the machine learning model configured to:
- receive audio data decoded according to a speech codec,
- determine a bitrate for the audio data based on a quantity of bits for a plurality of frames of the audio data,
- extract feature data from the audio data, and
- encode the feature data to produce one or more latent feature vectors according to the bitrate for the audio data,
- filter the audio data according to a plurality of filter coefficients that are dynamically adjusted on a per-frame basis using the one or more latent feature vectors, and
- provide the filtered audio data as speech-enhanced audio data.
A method, comprising:
- receiving audio data decoded according to a speech codec;
- processing the audio data through a deep neural network (DNN) machine learning model trained to enhance speech, wherein the DNN machine learning model:
- extracts feature data from the audio data,
- determines a bitrate for the audio data based on a quantity of bits received for a plurality of frames of the audio data,
- encodes the feature data as one or more latent feature vectors according to the bitrate for the audio data,
- filters, using at least one comb filter and at least one convolution filter, the audio data according to a plurality of filter coefficients that are dynamically adjusted on a per-frame basis using the one or more latent feature vectors, and
- generates speech-enhanced audio data based, at least in part, on the filtering of the audio data.
One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices of cause the one or more computing devices to implement:
- receiving audio data decoded according to a speech codec;
- causing the audio data to be processed through a deep neural network (DNN) machine learning model trained to enhance speech, wherein the DNN machine learning model:
- extracts feature data from the audio data,
- determines a bitrate for the audio data based on a quantity of bits received for a plurality of frames of the audio data,
- encodes the feature data as one or more latent feature vectors according to the bitrate for the audio data,
- filters the audio data according a plurality of filter coefficients that are dynamically adjusted on a per-frame basis using the one or more latent feature vectors, and
- generates speech-enhanced audio data as a result of filtering the audio data.
Dependencies are inferred from the claim text.