US 2020/0395013 A1Published application

Network Microphone Device With Command Keyword Conditioning

AssigneeSonos, Inc.

InventorsConnor Smith, John Tolomei, Kurt Soto

Life of this patent

Estimating the term…

Loading patent…

Abstract

In one aspect, a playback device includes a voice assistant service (VAS) wake-word engine and a command keyword engine. The playback device detects, via the command keyword engine, a first command keyword, and determines whether one or more playback conditions corresponding to the first command keyword are satisfied. Based on (a) detecting the first command keyword and (b) determining that the one or more playback conditions corresponding to the first command keyword are satisfied, the playback device playback device performs a first playback command corresponding to the first command keyword. When the playback device detects, via the wake-word engine, a wake-word in voice input, the playback device streams sound data corresponding to at least a portion of the voice input to one or more remote servers associated with the VAS.

Claims21

Dependent claims

A playback device comprising:

a network interface;
one or more microphones configured to detect sound;
at least one speaker;
one or more processors;
data storage having instructions stored thereon that are executable by the one or more processors to cause the playback device to perform functions comprising:
monitoring an input sound-data stream representing the sound detected by the one or more microphones for (i) a wake-word event and (ii) a media playback system keyword event;
detecting a first media playback system keyword event, wherein detecting the first media playback system keyword event comprises after detecting a first sound via the one or more microphones, determining, with at least a threshold confidence, that the detected first sound includes a first media playback system keyword, wherein the first media playback system keyword is one of a plurality of command keywords supported by the playback device;
in response to detecting the first media playback system keyword event, processing, via a local voice input engine of a media playback system voice assistant, the first sound as a first voice input, wherein processing the first sound comprises:
(i) determining that one or more media playback system keyword conditions corresponding to the first media playback system keyword are satisfied;
and
(ii) determining that local voice input engine is unable to determine an intent of the first voice input, wherein determining that local voice input engine is unable to determine the intent of the first voice input comprises determining that one or more parameter slots associated with the first media playback system keyword are not matched with keywords in the first voice input;
based on (a) detecting the first media playback system keyword event and (b) determining that local voice input engine is unable to determine the intent of the first voice input, sending, via the network interface, sound data corresponding to at least a portion of the first voice input to one or more servers of the media playback system voice assistant for processing of the first voice input;
after receiving data indicating one or more first playback operations according to an intent of the first voice input as determined by the one or more servers of the media playback system voice assistant, performing the one or more first playback operations;
detecting a first wake-word event, wherein detecting the first wake-word event comprises after detecting a second sound via the one or more microphones, determining that the detected second sound includes a second voice input comprising a first wake word;
and
in response to detecting the first wake-word event, streaming, via the network interface, sound data corresponding to at least a portion of the second voice input to one or more remote servers of a first voice assistant service.

13 more elements

A method to be performed by a playback device comprising a network interface and one or more microphones configured to detect sound, the method comprising:

monitoring an input sound-data stream representing the sound detected by the one or more microphones for (i) a wake-word event and (ii) a media playback system keyword event;
detecting a first media playback system keyword event, wherein detecting the first media playback system keyword event comprises after detecting a first sound via the one or more microphones, determining, with at least a threshold confidence, that the detected first sound includes a first media playback system keyword, wherein the first media playback system keyword is one of a plurality of command keywords supported by the playback device;
in response to detecting the first media playback system keyword event, processing, via a local voice input engine of a media playback system voice assistant, the first sound as a first voice input, wherein processing the first sound comprises:
(i) determining that one or more media playback system keyword conditions corresponding to the first media playback system keyword are satisfied;
and
(ii) determining that local voice input engine is unable to determine an intent of the first voice input, wherein determining that local voice input engine is unable to determine the intent of the first voice input comprises determining that one or more parameter slots associated with the first media playback system keyword are not matched with keywords in the first voice input;
based on (a) detecting the first media playback system keyword event and (b) determining that local voice input engine is unable to determine the intent of the first voice input, sending, via the network interface, sound data corresponding to at least a portion of the first voice input to one or more servers of the media playback system voice assistant for processing of the first voice input;
after receiving data indicating one or more first playback operations according to an intent of the first voice input as determined by the one or more servers of the media playback system voice assistant, performing the one or more first playback operations;
detecting a first wake-word event, wherein detecting the first wake-word event comprises after detecting a second sound via the one or more microphones, determining that the detected second sound includes a second voice input comprising a first wake word;
and
in response to detecting the first wake-word event, streaming, via the network interface, sound data corresponding to at least a portion of the second voice input to one or more remote servers of a first voice assistant service.

8 more elements

A tangible, non-transitory computer-readable medium having instructions stored thereon that are executable by one or more processors to cause a playback device to perform functions, the playback device comprising a network interface and one or more microphones configured to detect sound, the functions comprising:

monitoring an input sound-data stream representing the sound detected by the one or more microphones for (i) a wake-word event and (ii) a media playback system keyword event;
detecting a first media playback system keyword event, wherein detecting the first media playback system keyword event comprises after detecting a first sound via the one or more microphones, determining, with at least a threshold confidence, that the detected first sound includes a first media playback system keyword, wherein the first media playback system keyword is one of a plurality of command keywords supported by the playback device;
in response to detecting the first media playback system keyword event, processing, via a local voice input engine of a media playback system voice assistant, the first sound as a first voice input, wherein processing the first sound comprises:
(i) determining that one or more media playback system keyword conditions corresponding to the first media playback system keyword are satisfied;
and
(ii) determining that local voice input engine is unable to determine an intent of the first voice input, wherein determining that local voice input engine is unable to determine the intent of the first voice input comprises determining that one or more parameter slots associated with the first media playback system keyword are not matched with keywords in the first voice input;
based on (a) detecting the first media playback system keyword event and (b) determining that local voice input engine is unable to determine the intent of the first voice input, sending, via the network interface, sound data corresponding to at least a portion of the first voice input to one or more servers of the media playback system voice assistant for processing of the first voice input;
after receiving data indicating one or more first playback operations according to an intent of the first voice input as determined by the one or more servers of the media playback system voice assistant, performing the one or more first playback operations;
detecting a first wake-word event, wherein detecting the first wake-word event comprises after detecting a second sound via the one or more microphones, determining that the detected second sound includes a second voice input comprising a first wake word;
and
in response to detecting the first wake-word event, streaming, via the network interface, sound data corresponding to at least a portion of the second voice input to one or more remote servers of a first voice assistant service.

8 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/812,758Filed 9 Mar 2020
Priority: 12 Jun 2019
Published: 17 Dec 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

G10L 15/22
Speech recognition: procedures used during a speech recognition process, e.g. man-machine dialogue
G06F 3/165
Sound input; Sound output: management of the audio stream, e.g. setting of volume, audio stream path
G06F 3/167
Sound input; Sound output: audio in a user interface, e.g. using voice commands for navigating, audio feedback
G10L 15/08
Speech recognition: speech classification or search

Family

Application16/439,009Continuation · filed 12 Jun 2019
Application16/812,758Continuation
US 10,586,540 B1Related patent
US 11,501,773 B2Related patent
US 2020/0395013 A1This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key