US 12,190,902 B1Granted

Adaptive multi-stage output gain

AssigneeAmazon Technologies, Inc.

InventorsAditya Sharadchandra Joshi, Zhouhui Miao

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system configured to perform audio processing with adaptive multi-stage output gains. For example, an Audio Front End (AFE) component may generate a first output using a fixed gain value in order to improve device arbitration and a second output using an adaptive gain value in order to improve wakeword detection. A wakeword engine may process the second output to determine that a wakeword is present along with start/end times of the wakeword. The AFE component can use the start/end times to determine an amount of wakeword energy represented in the first output, which is sent to a remote device for device arbitration. The AFE component can also use the start/end times to determine an amount of wakeword energy represented in the second output, which can be used to determine the adaptive gain value that is unique to the device.

Claims20

Dependent claims

A computer-implemented method, the method comprising:

determining, by a first component of a first device, first audio data representing speech input received from at least one microphone of the first device;
performing, by the first component, audio processing to the first audio data to generate second audio data;
applying a first gain value to the second audio data to generate third audio data;
determining, using the third audio data, first energy data including a first energy value corresponding to a first audio frame of the third audio data, the first audio frame associated with a first timestamp;
applying a second gain value to the third audio data to generate fourth audio data;
encoding the fourth audio data with timestamp data to generate encoded fourth audio data, wherein a second audio frame of the encoded fourth audio data is associated with the first timestamp;
sending, from the first component to a second component, the encoded fourth audio data;
receiving, by the first component from the second component, wakeword boundary data including a second timestamp corresponding to a beginning of a wakeword and a third timestamp corresponding to an end of the wakeword;
selecting, using the second timestamp and the third timestamp, a portion of the first energy data that corresponds to the wakeword;
determining, using the portion of the first energy data, a second energy value representing an amount of energy corresponding to the wakeword;
determining, by a remote system, that the second energy value is greater than a third energy value associated with a second device;
and
in response to the second energy value being greater than the third energy value, causing, by the remote system, the first device to perform an action.

10 more elements

A computer-implemented method, the method comprising:

determining, by a device, audio data representing speech input received by the device;
generating, by the device, first modified audio data using the audio data and a first gain value;
generating, by the device, second modified audio data based at least in part on the audio data and a second gain value, the second gain value corresponding to wakeword detection;
determining, by the device and using the second modified audio data, a portion of the second modified audio data that represents a wakeword;
determining, by the device and based on the determining of the portion of the second modified audio data that represents the wakeword, a portion of the first modified audio data corresponding to the wakeword;
determining, by the device, first data representing one or more characteristics of the portion of the first modified audio data;
and
sending, by the device, the first data to a remote device.

5 more elements

A system comprising:

at least one processor;
and
memory including instructions operable to be executed by the at least one processor to cause the system to:
determine, by a device, audio data representing speech input received by the device;
generate, by the device, first modified audio data using the audio data and a first gain value;
generate, by the device, second modified audio data based at least in part on the audio data and a second gain value, the second gain value corresponding to wakeword detection;
determine, by the device and using the second modified audio data, a portion of the second modified audio data that represents a wakeword;
determine, by the device, a portion of the first modified audio data corresponding to the portion of the second modified audio data;
determine, by the device, first data representing one or more characteristics of the portion of the first modified audio data;
and
send, by the device, the first data to a remote device.

8 more elements

Dependencies are inferred from the claim text.