US 12,190,877 B1Granted

Device arbitration for speech processing

AssigneeAmazon Technologies, Inc.

InventorsJarred Barber, Tao Zhang, Yifeng Fan

Life of this patent

Estimating the term…

Loading patent…

Abstract

Devices and techniques are generally described for nearest device arbitration. In various examples, a first device may receive first audio data representing a wakeword spoken by a first speaker at a first time. In some examples, a second device may receive second audio data representing the wakeword spoken by the first speaker at the first time. In some cases, the first device may generate first feature data representing the first audio data and the second device may generate second feature data representing the second audio data. In various examples, a machine learning model may use the first feature data and the second feature data to generate first prediction data representing a prediction that the first device is closer to the first speaker than the second device.

Claims20

Dependent claims

A computer-implemented method comprising:

determining room size data by sampling a length l from a first distribution and a width w from a second distribution, wherein the room size data represents a room having dimensions including l×w;
determining a first location of a speaker in the room, the first location defined by a first two-dimensional coordinate, wherein the first location is sampled from a third distribution;
determining a second location of a noise source in the room, the second location defined by a second two-dimensional coordinate, wherein the second location is sampled from a fourth distribution;
determining a third location of a first speech processing-enabled device in the room, the third location defined by a third two-dimensional coordinate, wherein the third location is sampled from a fifth distribution;
determining a first audio sample representing a spoken wake word for the first speech processing-enabled device, wherein the first audio sample is sampled from a dataset of audio samples;
determining a second audio sample representing background noise for the noise source, wherein the second audio sample is sampled from a dataset of noise samples;
generating first audio data by convolving the first audio sample with a first impulse response associated with the first speech processing-enabled device, the first impulse response being associated with audio received at the third location from the first location;
generating second audio data by convolving the second audio sample with a second impulse response associated with the first speech processing-enabled device, the second impulse response being associated with audio received at the third location from the second location;
generating third audio data by adding the first audio data and the second audio data;
generating, by a deep learning model, first feature representation data representing the third audio data;
and
generating, by a classifier of the deep learning model using the first feature representation data, first prediction data indicating that the first speech processing-enabled device is closest to the first location among other speech processing-enabled devices.

9 more elements

A method comprising:

receiving, by a first device, first audio data representing sound at a first time, the first audio data being detected by one or more microphones of the first device;
receiving, by a second device, second audio data representing the sound at the first time, the second audio data being detected by one or more microphones of the second device;
generating, by a first encoder associated with the first device, first feature data representing the first audio data based at least in part on a first impulse response associated with the first device;
generating, by a second encoder associated with the second device, second feature data representing the second audio data based at least in part on a second impulse response associated with the second device;
and
generating, by a machine learning model using the first feature data and the second feature data, first prediction data representing a prediction that the first device is closer to a source of the sound than the second device.

3 more elements

A system comprising:

a first device;
a second device;
at least one processor;
and
non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to:
receive, by the first device, first audio data representing sound generated at a first time, the first audio data being detected by one or more microphones of the first device;
receive, by the second device, second audio data representing the sound generated at the first time, the second audio data being detected by one or more microphones of the second device;
generate, by a first encoder associated with the first device, first feature data representing the first audio data based at least in part on a first impulse response associated with the first device;
generate, by a second encoder associated with the second device, second feature data representing the second audio data based at least in part on a second impulse response associated with the second device;
and
generate, by a machine learning model using the first feature data and the second feature data, first prediction data representing a prediction that the first device is closer to a source of the sound than the second device.

8 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/685,232Filed 2 Mar 2022 · us provisional application
Priority: 9 Dec 2021
Granted: 7 Jan 2025
Term ends: Not estimatedThe source records don’t support an estimate.

Classification13 codes

G10L 15/22
Speech recognition: procedures used during a speech recognition process, e.g. man-machine dialogue
G10L 15/02
Speech recognition: feature extraction for speech recognition; Selection of recognition unit
G10L 15/16
Speech classification or search: using artificial neural networks
G10L 15/28
Speech recognition: constructional details of speech recognition systems

Family

Application63/287,687Us provisional application · filed 9 Dec 2021
US 12,190,877 B1This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key