US 10,373,612 B2Granted

Anchored speech detection and speech recognition

AssigneeAmazon Technologies, Inc.

InventorsSree Hari Krishnan Parthasarathi, Bjorn Hoffmeister, Brian King, Roland Maas

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system configured to process speech commands may classify incoming audio as desired speech, undesired speech, or non-speech. Desired speech is speech that is from a same speaker as reference speech. The reference speech may be obtained from a configuration session or from a first portion of input speech that includes a wakeword. The reference speech may be encoded using a recurrent neural network (RNN) encoder to create a reference feature vector. The reference feature vector and incoming audio data may be processed by a trained neural network classifier to label the incoming audio data (for example, frame-by-frame) as to whether each frame is spoken by the same speaker as the reference speech. The labels may be passed to an automatic speech recognition (ASR) component which may allow the ASR component to focus its processing on the desired speech.

Claims17

Dependent claims

A computer implemented method for identifying speech from a desired speaker for automatic speech recognition (ASR), the method comprising:

receiving audio data corresponding to speech, the audio data comprising a plurality of audio frames;
processing the plurality of audio frames to determine a first plurality of feature vectors corresponding to a first portion of the audio data and a second plurality of feature vectors corresponding to a second portion of the audio data;
determining that the first plurality of feature vectors corresponds to a wakeword;
processing the first plurality of feature vectors with a recurrent neural network encoder to determine a reference feature vector corresponding to speech from a desired speaker;
processing the second plurality of feature vectors, and the reference feature vector, using a neural-network classifier to determine a first score corresponding to a first feature vector in the second plurality, the first score corresponding to a likelihood that the first feature vector corresponds to audio spoken by the desired speaker;
determining that the score is above a threshold;
creating an indication that the first feature vector corresponds to speech from the desired speaker;
determining a first weight corresponding to the first feature vector based on the first feature vector corresponding to speech from the desired speaker;
and
performing ASR using the first weight and the first feature vector.

7 more elements

A computing system comprising:

at least one processor;
and
at least one memory including instructions that, when executed by the at least one processor, cause the computing system to:
receive first audio data as part of a first interaction with a device, the first audio data corresponding to first speech from a first speaker;
determine, using the first audio data and a recurrent neural network, a reference feature vector;
receiving second audio data as part of a second interaction with the device;
determine, using the reference feature vector and a trained model, that a first portion of the first audio data corresponds to second speech from a second speaker;
determine, using the reference feature vector and the trained model, that a second portion of the second audio data corresponds to third speech from the first speaker;
and
based at least in part on determining that the first portion of the second audio data corresponds to the second speaker, execute a command corresponding to the second portion of the second audio data.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/196,228Filed 29 Jun 2016 · us provisional application
Priority: 21 Mar 2016
Published: US 2017/0270919 A121 Sep 2017
Granted: 6 Aug 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification11 codes

G10L 15/20
Speech recognition: speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech
G10L 15/02
Speech recognition: feature extraction for speech recognition; Selection of recognition unit
G10L 15/08
Speech recognition: speech classification or search
G10L 15/16
Speech classification or search: using artificial neural networks

Family

Application62/311,082Us provisional application · filed 21 Mar 2016
US 2017/0270919 A1Pre-grant publication · published 21 Sep 2017
US 10,373,612 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key