US 2020/0349928 A1 · DEEP MULTI-CHANNEL ACOUSTIC MODELING

Abstract

Techniques for speech processing using a deep neural network (DNN) based acoustic model front-end are described. A new modeling approach directly models multi-channel audio data received from a microphone array using a first model (e.g., multi-channel DNN) that takes in raw signals and produces a first feature vector that may be used similarly to beamformed features generated by an acoustic beamformer. A second model (e.g., feature extraction DNN) processes the first feature vector and transforms it to a second feature vector having a lower dimensional representation. A third model (e.g., classification DNN) processes the second feature vector to perform acoustic unit classification and generate text data. These three models may be jointly optimized for speech processing (as opposed to individually optimized for signal enhancement), enabling improved performance despite a reduction in microphones and a reduction in bandwidth consumption during real-time processing.

Claims20

Dependent claims

A computer-implemented method, the method comprising:

receiving input audio data, the input audio data corresponding to at least a first microphone associated with a device and a second microphone associated with the device;
processing the input audio data using a first machine learning model to generate first data, a first portion of the first data generated based on a first power value associated with the input audio data and a second power value associated with the input audio data;
processing the first data using a second machine learning model to generate second data, a first portion of the second data corresponding to a first frequency range, a second portion of the second data corresponding to a second frequency range;
and
sending the second data to one or more speech processing components.

2 more elements

A system comprising:

at least one processor;
and
memory including instructions operable to be executed by the at least one processor to cause the system to:
receive input audio data, the input audio data corresponding to at least a first microphone associated with a device and a second microphone associated with the device;
process the input audio data using a first machine learning model to generate first data, a first portion of the first data generated based on a first power value associated with the input audio data and a second power value associated with the input audio data;
process the first data using a second machine learning model to generate second data, a first portion of the second data corresponding to a first frequency range, a second portion of the second data corresponding to a second frequency range;
and
send the second data to one or more speech processing components.

5 more elements

Dependencies are inferred from the claim text.

Deep multi-channel acoustic modeling

Life of this patent

Deep multi-channel acoustic modeling

Life of this patent

Abstract

Claims20