# US 2022/0036896 A1 · Hybrid voice command technique utilizing both on-device and cloud resources

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036896 A1
- Document ID: US20220036896A1
- Status: Published application
- Kind: A1
- Application number: 17/275,746
- Filed: 2019-09-13
- Published: 2022-02-03
- Earliest claimed: 2018-09-14
- Assignees: Aondevices, Inc.
- Applicants: Aondevices, Inc.
- Inventors: Mouna Elkhatib; Adil Benyassine
- Web page: https://patentgrep.com/patent/US20220036896A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06N 3/02: Neural networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G10L 15/08: Speech classification or search
- G10L 15/16: using artificial neural networks
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 25/30: using neural networks
- G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10)
- H03M 1/12: Analogue/digital converters ({H03M1/001 – } H03M1/10 take precedence)
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)

## Abstract

Systems and methods are presented for recognizing and responding to voice commands at a local system and selectively streaming audio to a network-based computing system to recognize voice commands when the user provides a specific voice command to stream to the network-based computing system and/or when the user provides a voice command that is not recognizable by the local system.

## Claims

### Claim 1

An electronic device, comprising:
a non-transitory computer readable storage medium having information stored thereon, the information comprising a plurality of command phrases,
wherein each of the plurality of command phrases is associated with a respective command instruction;
a network transceiver; and
an electrical circuit in communication with the storage medium and the network transceiver, the electrical circuit configured to
receive an audio signal,
monitor the audio signal to determine a presence of at least one of the command phrases, and
provide the respective command instruction associated with the detected command phrase,
wherein a first command phrase of the plurality of command phrases is associated with a respective command instruction to cause the network transceiver to transmit the audio signal, and
wherein a second command phrase of the plurality of command phrases is associated with a respective command instruction configured to executed by the portable electronic device without transmission of the audio signal.

### Claim 2

(canceled)

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein the portable electronic device is an “always-listening” device.

### Claim 4

(canceled)

### Claim 5

(canceled)

### Claim 6 (depends on 1)

The electronic device of claim 1,
wherein the information further comprises a wake word, and
wherein the electrical circuit is further configured to monitor the audio signal to determine a presence of the wake word.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein a third command phrase of the plurality of command phrases is associated with a respective command instruction to cause the network transceiver to cease transmitting the audio signal.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein the electrical circuit comprises a digital signal processor.

### Claim 9 (depends on 1)

The electronic device of claim 1, wherein the electrical circuit comprises a neural network circuit.

### Claim 10 (depends on 9)

The electronic device of claim 9, wherein the neural network comprises one or more of a convolutional neural network and/or a recurrent neural network.

### Claim 11 (depends on 1)

The electronic device of claim 1, wherein the electrical circuit comprises:
a voice activity detection (VAD) module configured to monitor the audio signal to determine a presence of a voice;
a keyword spotting module in communication with the VAD module and the memory store, the keyword spotting module configured to activate in response to the presence of the voice in the audio signal determined by the VAD and, while activated, monitor the audio signal to determine a presence of the wake word; and
a detection and recognition of commands module in communication with the memory store, the network transceiver, and the keyword spotting module, the recognition of commands module configured to activate in response to the presence of the wake word determined by the keyword spotting module and, while activated, monitor the audio signal to determine a presence of a command phrase of the plurality of predetermined command phrases, provide a command instruction associated with the detected command phrase.

### Claim 12 (depends on 11)

The electronic device of claim 11, wherein, while the electronic device is powered on, the audio signal is continuously being monitored by the VAD, monitored by the keyword spotting module, monitored by the detection and recognition of commands module, and/or transmitted by the transceiver.

### Claim 13 (depends on 11)

The electronic device of claim 11, wherein the keyword spotting module is further configured to deactivate at a first predetermined time period from a time at which the keyword spotting module is activated.

### Claim 14

(canceled)

### Claim 15 (depends on 11)

The electronic device of claim 11, wherein the detection and recognition of commands module is further configured to deactivate at a second predetermined time period from a time at which the detection and recognition of commands module is activated.

### Claim 16

(canceled)

### Claim 17

A system comprising:
a non-transitory computer readable storage medium having information stored thereon, the information comprising a plurality of command phrases and a wake word,
wherein each of the plurality of command phrases is associated with a respective command instruction;
a network transceiver; and
an electrical circuit in communication with the storage medium and the network transceiver, the electrical circuit configured to receive an audio signal,
monitor the audio signal to for a predetermined time period,
compare the audio signal during the predetermined time period to the plurality of command phrases, and
based on the comparison, either provide a command instruction associated with a command phrase of the plurality of command phrases or transmit the audio input signal monitored by the detection and recognition of commands module during the predetermined time period to the network transceiver.

### Claim 18

(canceled)

### Claim 19

(canceled)

### Claim 20

(canceled)

### Claim 21 (depends on 17)

The system of claim 17, wherein the electrical circuit is configured to begin monitoring the audio signal in response to powering on the electrical circuit and continuously monitor the audio signal while the electrical circuit is powered on.

### Claim 22

(canceled)

### Claim 23

(canceled)

### Claim 24

(canceled)

### Claim 25 (depends on 17)

The system of claim 17, wherein the electrical circuit comprises a neural network circuit, the neural network comprises one or more of a convolutional neural network and/or a recurrent neural network.

### Claim 26

(canceled)

### Claim 27 (depends on 17)

The system of claim 17, wherein electrical circuit comprises:
a voice activity detection (VAD) module configured to monitor the audio input signal to determine a presence of a voice in the audio input signal;
a keyword spotting module in communication with the VAD module and the memory store, the keyword spotting module configured to activate in response to the presence of the voice in the audio input signal determined by the VAD and, while activated, monitor the audio input signal to determine a presence of the wake word;
a detection and recognition of commands module in communication with the memory store, the network transceiver, and the keyword spotting module, the recognition of commands module configured to activate in response to the presence of the wake word determined by the keyword spotting module and, while activated, monitor the audio input signal to for the predetermined time period, compare the audio input signal during the predetermined time period to the plurality of command phrases, and based on the comparison, either provide the command instruction associated with the command phrase of the plurality of command phrases or transmit the audio input signal monitored by the detection and recognition of commands module during the predetermined time period to the network transceiver.

### Claim 28 (depends on 27)

The system of claim 27, wherein, while the system is powered on, the audio input signal is continuously being monitored by the VAD, monitored by the keyword spotting module, monitored by the detection and recognition of commands module, and/or transmitted by the transceiver.

### Claim 29 (depends on 27)

The system of claim 27, wherein the keyword spotting module is further configured to deactivate at a first predetermined time period from a time at which the keyword spotting module is activated.

### Claim 30

(canceled)

### Claim 31 (depends on 27)

The system of claim 27, wherein the detection and recognition of commands module is further configured to deactivate at a second predetermined time period from a time at which the detection and recognition of commands module is activated.

### Claim 32

(canceled)

### Claim 33

A method comprising:
monitoring, by an electronic device, an audio signal;
comparing, for a predetermined time period by the electronic device, the audio signal to a plurality of predetermined command phrases stored on the electronic device;
detecting, by the electronic device, a first command phrase of the plurality of command phrases in the audio signal;
based upon the comparison, executing, by the electronic device, a first command instruction associated with the first command phrase without the audio signal being transmitted from the electronic device;
detecting, by the electronic device, a second command phrase of the plurality of command phrases in the audio signal; and
based upon the comparison, transmitting, by the electronic device, to an external network, in response to detecting the second command phrase, the audio signal.

### Claim 34 (depends on 33)

The method of claim 33, further comprising:
monitoring, by the electronic device, the audio signal to determine a presence of a voice in the audio signal; and
monitoring, by the electronic device, in response to determining the presence of the voice in the audio signal, the audio signal to determine a presence of the wake word, and
comparing, by the electronic device, in response to determining the presence of the wake word, the audio signal to plurality of predetermined command phrases.

### Claim 35

(canceled)

### Claim 36

(canceled)