# US 2020/0349925 A1 · Online verification of custom wake word

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0349925 A1
- Document ID: US20200349925A1
- Status: Published application
- Kind: A1
- Application number: 16/522,401
- Filed: 2019-07-25
- Published: 2020-11-05
- Earliest claimed: 2019-05-05
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Khuram Shahid; Kshitiz Kumar; Teng Yi; Veljko Miljanic; Huaming Wang; Yifan Gong; Hosam Adel Khalil
- Web page: https://patentgrep.com/patent/US20200349925A1

## CPC

- G10L 15/08: Speech classification or search
- G10L 15/16: using artificial neural networks
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 17/24: the user being prompted to utter a password or a predefined phrase
- G10L 15/187: Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Generally discussed herein are devices, systems, and methods for wake word verification. A method can include receiving, at a server, a message from a device indicating that an utterance of a user-defined wake word was detected at the device, the message including (a) audio samples or features extracted from the audio samples and (b) data indicating the user-defined wake word, retrieving or generating, at the server, a custom decoding graph for the user-defined wake word, wherein the decoding graph and the static portion of the wake word verification model form a custom wake word verification model for the user-defined wake word, executing the wake word verification model to determine a likelihood that the wake word was uttered, and providing a message to the device indicating whether wake was uttered based on the determined likelihood.

## Claims

### Claim 1

A system comprising:
memory having stored model parameters that define a static portion of a wake word verification model; and
processing circuitry to:
receive a message from a device indicating that an utterance of a user-defined wake word was detected at the device, the message including (a) audio samples or features extracted from the audio samples and (b) data indicating the user-defined wake word;
retrieve or generate a custom decoding graph for the user-defined wake word, wherein the decoding graph and the static portion of the wake word verification model form a custom wake word verification model for the user-defined wake word;
execute the wake word verification model to determine a likelihood that the wake word was uttered; and
provide a message to the device indicating whether wake was uttered based on the determined likelihood.

### Claim 2 (depends on 1)

The system of claim 1, wherein the wake word verification model includes a prediction network that includes a decoding graph for only the wake word and a background language model (BLM).

### Claim 3 (depends on 2)

The system of claim 2, wherein the wake word is one of a plurality of wake words for the device and the decoding graph includes only the plurality of wake words and alternate pronunciations thereof.

### Claim 4 (depends on 2)

The system of claim 2, wherein the wake word verification model further includes a general acoustic model that operates on the audio samples or the features extracted from the audio samples to predict a series of phonemes corresponding thereto.

### Claim 5 (depends on 3)

The system of claim 3, wherein the acoustic model includes a plurality of neural network layers quantized to a specified number of bits.

### Claim 6 (depends on 5)

The system of claim 5, wherein an input neural network layer and an output neural network layer of the neural network layers are quantized to 16-bits and the remaining neural network layers are quantized to 8-bits.

### Claim 7 (depends on 4)

The system of claim 4, wherein the wake word verification model further includes a beam search decoder to determine, based on the predicated series of phonemes and a union of an output of the decoding graph and the BLM, a confidence that the wake word was uttered.

### Claim 8 (depends on 1)

The system of claim 1, wherein the wake word verification model is to determine that the wake word is present only in response to determining that the wake word was predicted to be uttered in at least two intermediate hypotheses since the last silence.

### Claim 9 (depends on 1)

The system of claim 1, wherein the processing circuitry is further to determine, in parallel, a task present in the audio samples or features extracted from the audio samples.

### Claim 10

A method of custom wake word verification comprising:
receiving, at a server, a message from a device indicating that an utterance of a user-defined wake word was detected at the device, the message including (a) audio samples or features extracted from the audio samples and (b) data indicating the user-defined wake word;
retrieving or generating, at the server, a custom decoding graph for the user-defined wake word, wherein the decoding graph and the static portion of the wake word verification model form a custom wake word verification model for the user-defined wake word;
executing the wake word verification model to determine a likelihood that the wake word was uttered; and
providing a message to the device indicating whether wake was uttered based on the determined likelihood.

### Claim 11 (depends on 10)

The method of claim 10, wherein the wake word verification model includes a prediction network that includes the custom decoding graph for only the user-specified wake word and a background language model (BLM).

### Claim 12 (depends on 11)

The method of claim 11, wherein the user-specified wake word is one of a plurality of wake words for the device and the custom decoding graph includes only the plurality of wake words and alternate pronunciations thereof.

### Claim 13 (depends on 12)

The method of claim 12, wherein the wake word verification model further includes a general acoustic model that operates on the audio samples or the features extracted from the audio samples to predict a series of phonemes corresponding thereto.

### Claim 14 (depends on 13)

The method of claim 13, wherein the acoustic model includes a plurality of neural network layers quantized to a specified number of bits.

### Claim 15 (depends on 14)

The method of claim 14, wherein an input neural network layer and an output neural network layer of the neural network layers are quantized to 16-bits and the remaining neural network layers are quantized to 8-bits.

### Claim 16

A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations for custom wake word verification, the operations comprising:
receiving a message from a device indicating that an utterance of a user-defined wake word was detected at the device, the message including (a) audio samples or features extracted from the audio samples and (b) data indicating the user-defined wake word;
retrieving or generating, at the server, a custom decoding graph for the user-defined wake word, wherein the decoding graph and the static portion of the wake word verification model form a custom wake word verification model for the user-defined wake word;
executing the wake word verification model to determine a likelihood that the wake word was uttered; and
providing a message to the device indicating whether wake was uttered based on the determined likelihood.

### Claim 17 (depends on 16)

The non-transitory machine-readable medium of claim 16, wherein the wake word verification model includes a prediction network that includes the custom decoding graph for only the user-specified wake word and a background language model (BLM).

### Claim 18 (depends on 17)

The non-transitory machine-readable medium of claim 17, wherein the wake word verification model further includes a general acoustic model that operates on the audio samples or the features extracted from the audio samples to predict a series of phonemes corresponding thereto.

### Claim 19 (depends on 18)

The non-transitory machine-readable medium of claim 18, wherein the wake word verification model further includes a beam search decoder to determine, based on the predicated series of phonemes and a union of an output of the decoding graph and the BLM, a confidence that the wake word was uttered.

### Claim 20 (depends on 16)

The non-transitory machine-readable medium of claim 16, wherein the operations further include determining, in parallel, a task present in the audio samples or features extracted from the audio samples.
