US 11,769,491 B1Granted

Performing utterance detection using convolution

AssigneeAmazon Technologies, Inc.

InventorsAbhishek Bafna, Haithem Albadawi

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system configured to perform utterance detection using data processing techniques that are similar to those used for object detection is provided. For example, the system may treat utterances within audio data as analogous to an object represented within an image and employ techniques to separate and identify individual utterances. The system may include one or more trained models that are trained to perform utterance detection. For example, the system may include a first module to process input audio data and identify whether speech is represented in the input audio data, a second module to apply convolution filters, and a third module configured to determine a boundary identifying a beginning and ending of a portion of the input audio data along with an utterance score indicating how closely the portion of the input audio data represents an utterance.

Claims21

Dependent claims

A computer-implemented method, the method comprising:

receiving first audio data corresponding to at least one microphone associated with a device;
performing feature extraction to generate first data representing audio features of the first audio data;
generating, using the first data and a first recurrent neural network (RNN), second data indicating first audio frames of the first audio data that represent speech and second audio frames of the first audio data that do not represent speech, the second data representing temporal features associated with the audio features;
processing, using at least one convolution layer that applies one or more convolution filters, the second data to generate third data, the third data representing spatial features associated with the audio features;
processing the third data using a trained model to generate fourth data, wherein the fourth data:
indicates that a first utterance is represented in a first portion of the first audio data,
indicates a beginning of the first portion of the first audio data,
indicates an ending of the first portion of the first audio data, and
includes a first score indicating a likelihood that the first portion of the first audio data represents the first utterance;
generating second audio data including the first portion of the first audio data;
and
causing speech processing to be performed on the second audio data.

9 more elements

A computer-implemented method, the method comprising:

receiving input audio data corresponding to at least one microphone associated with a device;
generating, using the input audio data, first data representing audio features corresponding to the input audio data;
generating, using the first data and a recurrent neural network (RNN), second data indicating first audio frames of the input audio data that represent speech;
generating, using the second data and one or more convolution filters, third data, the third data representing spatial features associated with the audio features;
and
processing the third data to generate fourth data, wherein the fourth data:
indicates that a first utterance is represented in a first portion of the input audio data,
indicates a beginning of the first portion of the input audio data,
indicates an ending of the first portion of the input audio data, and
includes a first score indicating a likelihood that the first portion of the input audio data represents the first utterance.

7 more elements

A system comprising:

at least one processor;
and
memory including instructions operable to be executed by the at least one processor to cause the system to:
receive input audio data corresponding to at least one microphone associated with a device;
generate, using the input audio data, first data representing audio features corresponding to the input audio data;
generate, using the first data and a recurrent neural network (RNN), second data indicating first audio frames of the input audio data that represent speech;
generate, using the second data and one or more convolution filters, third data, the third data representing spatial features associated with the audio features;
and
process the third data to generate fourth data, wherein the fourth data:
indicates that a first utterance is represented in a first portion of the input audio data,
indicates a beginning of the first portion of the input audio data,
indicates an ending of the first portion of the input audio data, and
includes a first score indicating a likelihood that the first portion of the input audio data represents the first utterance.

10 more elements

Dependencies are inferred from the claim text.