# US 2020/0395010 A1 · Network Microphone Device With Command Keyword Eventing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0395010 A1
- Document ID: US20200395010A1
- Status: Published application
- Kind: A1
- Application number: 16/439,032
- Filed: 2019-06-12
- Published: 2020-12-17
- Earliest claimed: 2019-06-12
- Assignees: Sonos, Inc.
- Applicants: Sonos, Inc.
- Inventors: Connor Smith; John Tolomei; Kurt Soto
- Web page: https://patentgrep.com/patent/US20200395010A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

In one aspect, a playback device includes a voice assistant service (VAS) wake-word engine and a command keyword engine. The playback device detects, via the command keyword engine, a first command keyword of in voice input of sound detected by one or more microphones of the playback device. The playback device determines an intent based on at least one keyword in the voice input via a local natural language unit (NLU). After detecting the first command keyword event and determining the intent, the playback device performs a first playback command corresponding to the first command keyword and according to the determined intent. When the playback device detects, via the wake-word engine, a wake-word in voice input, the playback device streams sound data corresponding to at least a portion of the voice input to one or more remote servers associated with the VAS.

## Claims

### Claim 1

A playback device comprising:
a network interface;
one or more processors;
at least one microphone configured to detect sound;
at least one speaker;
data storage having instructions stored thereon that are executable by the one or more processors to cause the playback device to perform functions comprising:
monitoring an input sound-data stream representing the sound detected by the at least one microphone for (i) a wake-word event and (ii) a first command keyword event;
detecting the wake-word event, wherein detecting the wake-word event comprises after detecting a first sound via the one or more microphones, determining that that the detected first sound includes a first voice input comprising a wake word;
streaming, via the network interface, sound data corresponding to at least a portion of the first voice input to one or more remote servers of a voice assistant service;
detecting the first command keyword event, wherein detecting the first command keyword event comprises after detecting a second sound via the one or more microphones, determining that the detected second sound includes a second voice input comprising a first command keyword and at least one keyword, wherein the first command keyword is one of a plurality of command keywords supported by the playback device, and wherein the first command keyword corresponds to a first playback command;
determining, via a local natural language unit (NLU), an intent based on the at least one keyword, wherein the NLU includes a pre-determined library of keywords comprising the at least one keyword; and
after (a) detecting the first command keyword event and (b) determining the intent, performing the first playback command according to the determined intent.

### Claim 2 (depends on 1)

The playback device of claim 1, wherein the functions further comprise:
detecting a second command keyword event, wherein detecting the second command keyword event comprises after detecting a third sound via the at least one microphone, determining that the third sound includes a third voice input comprising the second command keyword;
determining that the third voice input comprising the second command keyword does not include at least one other keyword from the pre-determined library of keywords; and
after determining that the third voice input comprising the second command keyword does not include the at one least keyword from the pre-determined library of keywords, streaming sound data representing at least a portion of the voice input comprising the second command keyword to one or more servers of the voice assistant service for processing by the one or more remote servers of the voice assistance service.

### Claim 3 (depends on 2)

The playback device of claim 2, wherein the functions further comprise:
playing back an audio prompt to request confirmation for invoking the voice assistant service to process the second command keyword; and
after playing back the audio prompt, receiving data representing confirmation to invoke the voice assistant service for processing the second command keyword, wherein
streaming the sound data representing at least a portion of the voice input comprising the second command keyword to one or more servers of the voice assistant service occurs only after receiving the data representing confirmation to invoke the voice assistant service.

### Claim 4 (depends on 1)

The playback device of claim 1, wherein the functions further comprise:
detecting a second command keyword event, wherein detecting the second command keyword event comprises after detecting a third sound via the at least one microphone, determining that the third sound includes a third voice input comprising the second command keyword;
determining that the third voice input comprising the second command keyword does not include at least one other keyword from the pre-determined library of keywords; and
after determining that the third voice input comprising the second command keyword does not include the at one least keyword from the pre-determined library of keywords, performing the first playback command according to one or more default parameters.

### Claim 5 (depends on 1)

The playback device of claim 1, wherein a first keyword of the at least one keyword in the detected second sound represents a zone name corresponding to a first zone of a media playback system, wherein performing the first playback command according to the determined intent comprises sending one or more instructions to perform the first playback command in the first zone, and wherein the media playback system comprises the playback device.

### Claim 6 (depends on 5)

The playback device of claim 5, wherein the functions further comprise:
populating the pre-determined library of keywords with zones names corresponding to respective zones within the media playback system, wherein each zone comprises one or more respective playback devices, and wherein the pre-determined library of keywords is populated with the zone name corresponding to the first zone of the media playback system.

### Claim 7 (depends on 1)

The playback device of claim 1, wherein the functions further comprise:
discovering, via the network interface, smart home devices connected to a local area network; and
populating the pre-determined library of keywords with names corresponding to respective smart home devices discovered on the local area network.

### Claim 8 (depends on 1)

The playback device of claim 1, wherein a media playback system comprises the playback device, wherein the media playback system is registered to one or more user profiles, and wherein the functions further comprise:
populating the pre-determined library of keywords with names corresponding to playlists that are designated as favorites by the one or more user profiles.

### Claim 9 (depends on 8)

The playback device of claim 8, wherein a first user profile of the one or more user profiles is associated with a user account of a first streaming audio service and a user account of a second streaming audio service, and wherein the playlists comprise a first playlist of a first streaming audio service designated as a favorite by the user account of the first streaming audio service and a second playlist comprising audio tracks from the first streaming audio service and the second streaming audio service.

### Claim 10 (depends on 1)

The playback device of claim 1, wherein detecting the first command keyword event further comprises determining that one or more playback conditions corresponding to the first command keyword are satisfied.

### Claim 11 (depends on 10)

The playback device of claim 10, wherein the one or more playback conditions corresponding to the first command keyword comprises a first condition of an absence of background speech in the detected first sound.

### Claim 12

A method to be performed by a playback device comprising at least one microphone configured to detect sound, the method comprising:
monitoring an input sound-data stream representing the sound detected by the at least one microphone for (i) a wake-word event and (ii) a first command keyword event;
detecting the wake-word event, wherein detecting the wake-word event comprises after detecting a first sound via the one or more microphones, determining that that the detected first sound includes a first voice input comprising a wake word;
streaming, via a network interface, sound data corresponding to at least a portion of the first voice input to one or more remote servers of a voice assistant service;
detecting the first command keyword event, wherein detecting the first command keyword event comprises after detecting a second sound via the one or more microphones, determining that the detected second sound includes a second voice input comprising a first command keyword and at least one keyword, wherein the first command keyword is one of a plurality of command keywords supported by the playback device, and wherein the first command keyword corresponds to a first playback command;
determining, via a local natural language unit (NLU), an intent based on the at least one keyword, wherein the NLU includes a pre-determined library of keywords comprising the at least one keyword; and
after (a) detecting the first command keyword event and (b) determining the intent, performing the first playback command according to the determined intent.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
detecting a second command keyword event, wherein detecting the second command keyword event comprises after detecting a third sound via the at least one microphone, determining that the third sound includes a third voice input comprising the second command keyword;
determining that the third voice input comprising the second command keyword does not include at least one other keyword from the pre-determined library of keywords; and
after determining that the third voice input comprising the second command keyword does not include the at one least keyword from the pre-determined library of keywords, streaming sound data representing at least a portion of the voice input comprising the second command keyword to one or more servers of the voice assistant service for processing by the one or more remote servers of the voice assistance service.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
playing back an audio prompt to request confirmation for invoking the voice assistant service to process the second command keyword; and
after playing back the audio prompt, receiving confirmation to invoke the voice assistant service for processing the second command keyword, wherein
streaming the sound data representing at least a portion of the voice input comprising the second command keyword to one or more servers of the voice assistant service occurs only after receiving the confirmation to invoke the voice assistant service.

### Claim 15 (depends on 12)

The method of claim 12, further comprising:
detecting a second command keyword event, wherein detecting the second command keyword event comprises after detecting a third sound via the at least one microphone, determining that the third sound includes a third voice input comprising the second command keyword;
determining that the third voice input comprising the second command keyword does not include at least one other keyword from the pre-determined library of keywords; and
after determining that the third voice input comprising the second command keyword does not include the at one least keyword from the pre-determined library of keywords, performing the first playback command according to one or more default parameters.

### Claim 16 (depends on 12)

The method of claim 12, wherein a first keyword of the at least one keyword in the detected second sound represents a zone name corresponding to a first zone of a media playback system of which the playback device forms a part, and wherein performing the first playback command according to the determined intent comprises sending one or more instructions to perform the first playback command in the first zone.

### Claim 17 (depends on 16)

The method of claim 16, wherein a media playback system comprises the playback device, the method further comprising:
populating the pre-determined library of keywords with zones names corresponding to respective zones within the media playback system, wherein each zone comprises one or more respective playback devices, and wherein the pre-determined library of keywords is populated with the zone name corresponding to the first zone of the media playback system.

### Claim 18 (depends on 12)

The method of claim 12, further comprising:
discovering, via the network interface, smart home devices connected to a local area network; and
populating the pre-determined library of keywords with names corresponding to respective smart home devices discovered on the local area network.

### Claim 19 (depends on 12)

The method of claim 12, wherein a media playback system comprises the playback device, wherein the media playback system is registered to one or more user profiles, and wherein the method further comprises:
populating the pre-determined library of keywords with names corresponding to playlists that are designated as favorites by the one or more user profiles.

### Claim 20

A non-transitory computer-readable medium having instructions stored thereon that are executable by one or more processors to cause a playback device to perform functions, the playback device comprising at least one microphone configured to detect sound, the functions comprising:
monitoring an input sound-data stream representing the sound detected by the at least one microphone for (i) a wake-word event and (ii) a first command keyword event;
detecting the wake-word event, wherein detecting the wake-word event comprises after detecting a first sound via the one or more microphones, determining that that the detected first sound includes a first voice input comprising a wake word;
streaming, via a network interface, sound data corresponding to at least a portion of the first voice input to one or more remote servers of a voice assistant service;
detecting the first command keyword event, wherein detecting the first command keyword event comprises after detecting a second sound via the one or more microphones, determining that the detected second sound includes a second voice input comprising a first command keyword and at least one keyword, wherein the first command keyword is one of a plurality of command keywords supported by the playback device, and wherein the first command keyword corresponds to a first playback command;
determining, via a local natural language unit (NLU), an intent based on the at least one keyword, wherein the NLU includes a pre-determined library of keywords comprising the at least one keyword; and
after (a) detecting the first command keyword event and (b) determining the intent, performing the first playback command according to the determined intent.
