Back to results
US 11,308,939 B1Granted
Wakeword detection using multi-word model
AssigneeAmazon Technologies, Inc.
InventorsYixin Gao, Ming Sun, Varun Nagaraja, Gengshen Fu, Chao Wang, Shiv Naga Prasad Vitaladevuni
Life of this patent
Estimating the term…
Abstract
A system and method performs wakeword detection and automatic speech recognition using the same acoustic model. A mapping engine maps phones/senones output by the acoustic model to phones/senones corresponding to the wakeword. A hidden Markov model (HMM) may determine that the wakeword is present in audio data; the HMM may have multiple paths for multiple wakewords or may have multiple models. Once the wakeword is detected, ASR is performed using the acoustic model.
Claims19
A computer-implemented method comprising:
- receiving, from at least one microphone, first audio data representing first audio, the first audio data representing a first wakeword and a first command following the first wakeword;
- determining, using a feature-extraction model, first audio features representing the first audio data;
- determining, using an acoustic model and the first audio features, first senone data representing a first set of senones in the first audio;
- sending the first senone data to a first component;
- prior to completion of processing by the first component related to the first senone data, sending the first senone data to a second component that does not receive output data from the first component;
- determining, using the first component and based at least in part on the first senone data, that the first set of senones includes a first subset of senones corresponding to the first wakeword;
- processing the first senone data using the second component to identify at least one sequence of senones corresponding to at least one word;
- determining that the at least one word corresponds to the first command;
- and
- executing the first command.
7 more elements
A computer-implemented method comprising:
- determining first audio characteristic data corresponding to first audio data, the first audio data representing a first wakeword and a first command following the first wakeword;
- determining, using an acoustic model and the first audio characteristic data, first data representing a first set of acoustic units represented in the first audio data;
- sending the first data to a first component;
- prior to completion of processing by the first component related to the first data, sending the first data to a second component that does not receive output data from the first component;
- determining, using the first component and based at least in part on the first data, that the first set of acoustic units includes a first subset of acoustic units corresponding to the first wakeword;
- processing the first data using the second component to identify at least one sequence of acoustic units corresponding to at least one word;
- determining that the at least one word corresponds to the first command;
- and
- executing the first command.
6 more elements
A system comprising:
- at least one processor;
- and
- at least one memory including instructions that, when executed by the at least one processor, cause the system to:
- determine first audio characteristic data corresponding to first audio data, the first audio data representing a first wakeword and a first command following the first wakeword,
- determine, using an acoustic model and the first audio characteristic data, first data representing a first set of acoustic units represented in the first audio data,
- send the first data to a first component,
- prior to completion of processing by the first component related to the first data, sending the first data to a second component that does not receive output data from the first component,
- determine, using the first component and based at least in part on the first data, that the first set of acoustic units includes a first subset of acoustic units corresponding to the first wakeword,
- process the first data using the second component to identify at least one sequence of acoustic units corresponding to at least one word,
- determine that the at least one word corresponds to the first command, and
- execute the first command.
8 more elements
Dependencies are inferred from the claim text.