# US 2024/0201938 A1 · Pairing audio data channels based on initiating, using a first client device, playback of media on a second client device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0201938 A1
- Document ID: US20240201938A1
- Status: Published application
- Kind: A1
- Application number: 18/083,281
- Filed: 2022-12-16
- Published: 2024-06-20
- Earliest claimed: 2022-12-15
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Dongeek Shin
- Web page: https://patentgrep.com/patent/US20240201938A1

## CPC

- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 2015/223: {Execution procedure of a spoken command}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Techniques disclosed herein are directed towards pairing a first audio data stream of a first client device with a second audio data stream of a second client device based on identifying a user has imitated playback of media at the second client device, where the imitation of the playback by the user occurs at the first client device. Many implementations include the user controlling the media playback at the second client device using one or more automated assistant queries. Various implementations include deleting the pairing between the first audio data stream and the second audio data stream in response to determining whether one or more conditions have been satisfied (e.g., whether the user stops the playback, whether the first client device is a threshold distance away from the second client device, whether a period of time has elapsed, whether the entire media has played at the second client device, whether one or more additional or alternative conditions are satisfied, and/or combinations thereof).

## Claims

### Claim 1

A method implemented by one or more processors, the method comprising:
determining, based on processing input received from a user of a first client device, to initiate playback of media at a second client device;
in response to determining to initiate the playback of the media at the second client device and in response to the input being received at the first client device,
generating, for a period of time, a pairing between a first audio data stream and a second audio data stream, the first audio data stream detected via one or more first microphones of the first client device and the second audio data stream detected via one or more second microphones of the second device; and
in response to determining the user invoked the first client device and/or the second client device, and based on the pairing:
processing an automated assistant query captured in the first audio data stream and/or the second audio data stream.

### Claim 2 (depends on 1)

The method of claim 1, wherein the automated assistant query is captured in an utterance spoken by the user subsequent to invoking the first client device and/or the second client device, and further comprising:
controlling the media playback on the second client device based on the automated assistant query.

### Claim 3 (depends on 2)

The method of claim 2, wherein the utterance is captured via the one or more first microphones of the first client device.

### Claim 4 (depends on 2)

The method of claim 2, wherein the utterance is not captured by the one or more second microphones of the second client device.

### Claim 5 (depends on 4)

The method of claim 4, wherein the utterance is not captured by the one or more second microphones of the second client device based on a distance between the user speaking the utterance and the one or more second microphones of the second client device.

### Claim 6 (depends on 4)

The method of claim 4, wherein the utterance is not captured by the one or more second microphones based on interference between an audio portion of the media and the utterance.

### Claim 7 (depends on 3)

The method of claim 3, wherein the utterance is additionally captured via one or more second microphones of the second client device.

### Claim 8 (depends on 2)

The method of claim 2, wherein the utterance is captured via the one or more second microphones of the second client device and the utterance is not captured at the one or more first microphones of the first client device.

### Claim 9 (depends on 1)

The method of claim 1, wherein generating, for the period of time, the pairing between the first audio data stream and the second audio data stream comprises generating a merged audio data stream by merging the first audio data stream and the second audio data stream via a communications channel, and wherein processing the automated assistant query captured in the first audio data stream and/or the second audio data stream comprises processing the automated assistant query captured in the merged audio data stream.

### Claim 10 (depends on 1)

The method of claim 1, wherein, for the period of time, the pairing between the first audio data stream and the second audio data stream comprises processing the first audio data stream in parallel with processing the second audio data stream, and wherein processing the automated assistant query captured in the first audio data stream and/or the second audio data stream comprises processing the first audio data stream in parallel with processing the second audio data stream.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
determining whether the playback of the media at the second client device has ceased based on determining whether one or more conditions are satisfied; and
in response to determining the playback of the media at the second client device has ceased, deleting the pairing between the first audio data stream and the second audio data stream.

### Claim 12 (depends on 11)

The method of claim 11, wherein determining whether the one or more conditions are satisfied includes determining whether the period of time has elapsed.

### Claim 13 (depends on 11)

The method of claim 11, wherein determining whether the one or more conditions are satisfied includes determining whether the user has ended the playback of the media.

### Claim 14 (depends on 11)

The method of claim 11, wherein determining whether the one or more conditions are satisfied includes determining whether the first client device is a threshold distance away from the second client device.

### Claim 15

A system comprising
memory storing instructions;
one or more processors operable to execute the instructions to:
determine, based on processing input received from a user of a first client device, to initiate playback of media at a second client device;
in response to determining to initiate the playback of the media at the second client device and in response to the input being received at the first client device,
generate, for a period of time, a pairing between a first audio data stream and a second audio data stream, the first audio data stream detected via one or more first microphones of the first client device and the second audio data stream detected via one or more second microphones of the second device; and
in response to determining the user invoked the first client device and/or the second client device, and based on the pairing:
process an automated assistant query captured in the first audio data stream and/or the second audio data stream.

### Claim 16 (depends on 15)

The system of claim 15, wherein the automated assistant query is captured in an utterance spoken by the user subsequent to invoking the first client device and/or the second client device, and wherein one or more of the processors are further operable to execute the instructions to:
control the media playback on the second client device based on the automated assistant query.

### Claim 17 (depends on 16)

The system of claim 16, wherein the utterance is captured via the one or more first microphones of the first client device.

### Claim 18 (depends on 16)

The system of claim 16, wherein the utterance is not captured by the one or more second microphones of the second client device.

### Claim 19 (depends on 18)

The system of claim 18, wherein the utterance is not captured by the one or more second microphones of the second client device based on a distance between the user speaking the utterance and the one or more second microphones of the second client device.

### Claim 20 (depends on 18)

The system of claim 18, wherein the utterance is not captured by the one or more second microphones based on interference between an audio portion of the media and the utterance.
