# US 2022/0093114 A1 · Key phrase detection with audio watermarking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0093114 A1
- Document ID: US20220093114A1
- Status: Published application
- Kind: A1
- Application number: 17/457,820
- Filed: 2021-12-06
- Published: 2022-03-24
- Earliest claimed: 2017-11-28
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Ricardo Antonio Garcia
- Web page: https://patentgrep.com/patent/US20220093114A1

## CPC

- G10L 19/018: Audio watermarking, i.e. embedding inaudible data in the audio signal
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 21/31: User authentication
- G10L 15/08: Speech classification or search
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 21/00: Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence)
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/8358: involving watermark {(protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467)}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using audio watermarks with key phrases. One of the methods includes receiving, by a playback device, an audio data stream; determining, before the audio data stream is output by the playback device, whether a portion of the audio data stream encodes a particular key phrase by analyzing the portion using an automated speech recognizer; in response to determining that the portion of the audio data stream encodes the particular key phrase, modifying the audio data stream to include an audio watermark; and providing the modified audio data stream for output.

## Claims

### Claim 1

A computer-implemented method that when executed on data processing hardware causes the data processing hardware to perform operations comprising:
determining whether an audio data stream to be output through a speaker encodes a key phrase, the audio data stream corresponding to one of music content or video content;
when the audio stream encodes the key phrase, creating a modified audio data stream by:
dynamically generating multiple audio watermarks encoding data that indicates the audio data stream originated from a content provider; and
inserting the dynamically generated multiple audio watermarks into the audio data stream to create the modified audio data stream; and
providing the modified audio data stream for output through the speaker.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein:
the data processing hardware resides on a playback device; and
prior to determining whether the audio data stream to be output through the speaker encodes the key phrase, the playback device receives the audio data stream from the content provider through a wireless input connection other than a microphone.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the playback device:
receives the audio data stream in a video stream from the content provider through the wireless input connection; and
connects to a display using a digital audio and video connection.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the operations further comprise, when providing the modified audio data stream for output through the speaker, providing, using the digital audio and video connection, a video portion of the video stream for presentation by the display.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein the playback device synchronizes presentation of the video portion of the video stream by the display with the modified audio data stream for output through the speaker.

### Claim 6 (depends on 3)

The computer-implemented method of claim 3, where the playback device connects to a television using the digital audio and video connection, the television comprising the display and the speaker.

### Claim 7 (depends on 2)

The computer-implemented method of claim 2, wherein the playback device comprises the speaker.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein after providing the modified data stream for output through the speaker, a listening device, while in an awake mode responsive to detecting a key phrase via a microphone, is configured to:
capture the modified audio data stream via the microphone; and
determine an action to perform using the multiple audio watermarks encoding the data that indicates the audio data stream originated from the content provider.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the listening device is located in a same room as the speaker.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein:
a portion of the multiple audio watermarks in the modified audio data stream encode different data than the other multiple audio watermarks; or
each of the multiple audio watermarks encode the same data.

### Claim 11

A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware and storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
determining whether an audio data stream to be output through a speaker encodes a key phrase, the audio data stream corresponding to one of music content or video content;
when the audio stream encodes the key phrase, creating a modified audio data stream by:
dynamically generating multiple audio watermarks encoding data that indicates the audio data stream originated from a content provider; and
inserting the dynamically generated multiple audio watermarks into the audio data stream to create the modified audio data stream; and
providing the modified audio data stream for output through the speaker.

### Claim 12 (depends on 11)

The system of claim 11, wherein:
the data processing hardware and the memory hardware reside on a playback device; and
prior to determining whether the audio data stream to be output through the speaker encodes the key phrase, the playback device receives the audio data stream from the content provider through a wireless input connection other than a microphone.

### Claim 13 (depends on 12)

The system of claim 12, wherein the playback device:
receives the audio data stream in a video stream from the content provider through the wireless input connection; and
connects to a display using a digital audio and video connection.

### Claim 14 (depends on 13)

The system of claim 13, wherein the operations further comprise, when providing the modified audio data stream for output through the speaker, providing, using the digital audio and video connection, a video portion of the video stream for presentation by the display.

### Claim 15 (depends on 14)

The system of claim 14, wherein the playback device synchronizes presentation of the video portion of the video stream by the display with the modified audio data stream for output through the speaker.

### Claim 16 (depends on 13)

The system of claim 13, where the playback device connects to a television using the digital audio and video connection, the television comprising the display and the speaker.

### Claim 17 (depends on 12)

The system of claim 12, wherein the playback device comprises the speaker.

### Claim 18 (depends on 11)

The system of claim 11, wherein after providing the modified data stream for output through the speaker, a listening device, while in an awake mode responsive to detecting a key phrase via a microphone, is configured to:
capture the modified audio data stream via the microphone; and
determine an action to perform using the multiple audio watermarks encoding the data that indicates the audio data stream originated from the content provider.

### Claim 19 (depends on 11)

The system of claim 11, wherein the listening device is located in a same room as the speaker.

### Claim 20 (depends on 11)

The system of claim 11, wherein:
a portion of the multiple audio watermarks in the modified audio data stream encode different data than the other multiple audio watermarks; or
each of the multiple audio watermarks encode the same data.
