# US 2020/0349927 A1 · On-device custom wake word detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0349927 A1
- Document ID: US20200349927A1
- Status: Published application
- Kind: A1
- Application number: 16/522,416
- Filed: 2019-07-25
- Published: 2020-11-05
- Earliest claimed: 2019-05-05
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Emilian Stoimenov; Rui Zhao; Kaustubh Prakash Kalgaonkar; Ivaylo Andreanov Enchev; Khuram Shahid; Anthony Phillip Stark; Guoli Ye; Mahadevan Srinivasan; Yifan Gong; Hosam Adel Khalil
- Web page: https://patentgrep.com/patent/US20200349927A1

## CPC

- G10L 15/063: {Training}
- G10L 15/16: using artificial neural networks
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G10L 17/24: the user being prompted to utter a password or a predefined phrase
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G10L 15/07: to the speaker
- G10L 2015/088: {Word spotting}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Generally discussed herein are devices, systems, and methods for on-device detection of a wake word. A device can include a memory including model parameters that define a custom wake word detection model, the wake word detection model including a recurrent neural network transducer (RNNT) and a lookup table (LUT), the LUT indicating a hidden vector to be provided in response to a phoneme of a user-specified wake word, a microphone to capture audio, and processing circuitry to receive the audio from the microphone, determine, using the wake word detection model, whether the audio includes an utterance of the user-specified wake word, and wake up a personal assistant after determining the audio includes the utterance of the user-specified wake word.

## Claims

### Claim 1

A device comprising:
memory including model parameters that define a custom wake word detection model, the wake word detection model including a recurrent neural network transducer (RNNT) and a lookup table (LUT), the LUT indicating a hidden vector to be provided in response to a phoneme of a user-specified wake word;
a microphone to capture audio;
processing circuitry to:
receive the audio from the microphone;
determine, using the wake word detection model, whether the audio includes an utterance of the user-specified wake word; and
wake up a personal assistant after determining the audio includes the utterance of the user-specified wake word.

### Claim 2 (depends on 1)

The device of claim 1, wherein the wake word detection model is trained using standard phonemes and whole word phonemes.

### Claim 3 (depends on 1)

The device of claim 1, wherein the processing circuitry is further to reset the wake word detection model to erase a history of processed audio.

### Claim 4 (depends on 3)

The device of claim 3, wherein the reset occurs in response to determining one of the wake word was uttered and a specified period of time has elapsed.

### Claim 5 (depends on 1)

The device of claim 1, wherein the wake word detection model is compressed using single value decomposition (SVD).

### Claim 6 (depends on 5)

The device of claim 5, wherein the wake word detection model includes weights quantized to 8-bit or 16-bit values.

### Claim 7 (depends on 1)

The device of claim 1, wherein the processing circuitry is to:
receive the wake word from a user;
provide the wake word to a wake word model engine; and
receive from the wake word model engine, a wake word graph of the wake word indicating a phoneme sequence of the wake word and alternate pronunciations of the wake word;
wherein the wake word graph is part of the wake word detection model.

### Claim 8 (depends on 1)

The device of claim 1, wherein the processing circuitry is to:
receive the wake word from a user;
provide the wake word to a wake word model engine; and
receive from the wake word model engine, a wake word graph of the wake word indicating a phoneme sequence of the wake word and alternate pronunciations of the wake word and a background language model with unigrams and bi-grams of the wake word removed therefrom;
wherein the wake word graph and the background language model are part of the wake word detection model.

### Claim 9

A method of on-device custom wake word detection comprising:
receiving audio from a microphone of a device;
determining, using a wake word detection model, whether the audio includes an utterance of a user-specified wake word, the wake word detection model including a recurrent neural network transducer (RNNT) and a lookup table (LUT), the LUT indicating a hidden vector to be provided in response to a phoneme of a user-specified wake word; and
waking up a personal assistant after determining the audio includes the utterance of the user-specified wake word.

### Claim 10 (depends on 9)

The method of claim 9, wherein the wake word detection model is trained using standard phonemes and whole word phonemes.

### Claim 11 (depends on 9)

The method of claim 9, further comprising resetting the wake word detection model to erase a history of processed audio.

### Claim 12 (depends on 11)

The method of claim 11, wherein the reset occurs in response to determining one of the wake word was uttered and a specified period of time has elapsed.

### Claim 13 (depends on 9)

The method of claim 9, wherein the wake word detection model is compressed using single value decomposition (SVD).

### Claim 14 (depends on 13)

The method of claim 13, wherein the wake word detection model includes weights quantized to 8-bit or 16-bit values.

### Claim 15 (depends on 9)

The method of claim 9, further comprising:
receiving the user-specified wake word from a user;
providing the user-specified wake word to a wake word model engine; and
receiving from the wake word model engine, a wake word graph of the user-specified wake word indicating a phoneme sequence of the user-specified wake word and alternate pronunciations of the user-specified wake word;
wherein the wake word graph is part of the wake word detection model.

### Claim 16

A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations for on-device custom wake word detection, the operations comprising:
receiving audio from a microphone of a device;
determining, using a wake word detection model, whether the audio includes an utterance of a user-specified wake word, the wake word detection model including a recurrent neural network transducer (RNNT) and a lookup table (LUT), the LUT indicating a hidden vector to be provided in response to a phoneme of a user-specified wake word; and
waking up a personal assistant after determining the audio includes the utterance of the user-specified wake word.

### Claim 17 (depends on 16)

The non-transitory machine-readable medium of claim 16, wherein the wake word detection model is trained using standard phonemes and whole word phonemes.

### Claim 18 (depends on 16)

The non-transitory machine-readable medium of claim 16, wherein the operations further comprise resetting the wake word detection model to erase a history of processed audio in response to determining one of the wake word was uttered and a specified period of time has elapsed.

### Claim 19 (depends on 16)

The non-transitory machine-readable medium of claim 16, wherein the operations further comprise:
receiving the user-specified wake word from a user;
providing the user-specified wake word to a wake word model engine; and
receiving from the wake word model engine, a wake word graph of the user-specified wake word indicating a phoneme sequence of the user-specified wake word and alternate pronunciations of the user-specified wake word;
wherein the wake word graph is part of the wake word detection model.

### Claim 20 (depends on 16)

The non-transitory machine-readable medium of claim 16, wherein the operations further include:
receiving the wake word from a user;
providing the wake word to a wake word model engine; and
receiving from the wake word model engine, a wake word graph of the wake word indicating a phoneme sequence of the wake word and alternate pronunciations of the wake word and a background language model with unigrams and bi-grams of the wake word removed therefrom;
wherein the wake word graph and the background language model are part of the wake word detection model.
