# US 12,014,738 B2 · Arbitrating between multiple potentially-responsive electronic devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,014,738 B2
- Document ID: US12014738B2
- Status: Granted
- Kind: B2
- Application number: 18/144,713
- Filed: 2023-05-08
- Published: 2023-08-31
- Granted: 2024-06-18
- Earliest claimed: 2020-09-02
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Garrett Frederick Berg; Zac Livingston
- Web page: https://patentgrep.com/patent/US12014738B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 15/28: Constructional details of speech recognition systems
- H04S 7/301: {Automatic calibration of stereophonic sound system, e.g. with test microphone}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04S: STEREOPHONIC SYSTEMS

## Abstract

Techniques described herein are directed to arbitrating between multiple potentially-responsive, automated-assistant capable electronic devices to determine which should respond to the user's utterance, and/or which should defer to other electronic device(s). In various implementations, a spoken utterance of a user may be detected at a microphone of a first electronic device, a spoken utterance provided by a user. Sound(s) emitted by additional electronic device(s) may also be detected at the microphone. Each of the sound(s) may encode a timestamp corresponding to detection of the spoken utterance at a respective electronic device. Timestamp(s) may be extracted from the sound(s) and compared to a local timestamp corresponding to detection of the spoken utterance at the first electronic device. Based on the comparison, the first electronic device may either invoke an automated assistant locally or defer to one of the additional electronic devices.

## Claims

### Claim 1

A method implemented using one or more processors of a first electronic device, comprising:
detecting, at a microphone of the first electronic device, a spoken utterance provided by a user;
detecting one or more light emissions emitted by one or more additional electronic devices at a light detector of the first electronic device, wherein each of the one or more light emissions encodes one or more timestamps corresponding to detection of the spoken utterance at a respective electronic device of the one or more additional electronic devices;
extracting, from the one or more light emissions emitted by the one or more additional electronic devices, the one or more timestamps;
comparing the one or more timestamps to a local timestamp corresponding to detection of the spoken utterance at the first electronic device; and
based on the comparing, causing the first electronic device to either invoke an automated assistant locally or defer to one of the additional electronic devices.

### Claim 2 (depends on 1)

The method of claim 1, further comprising causing the first electronic device to emit another light emission, wherein the another light emission encodes the local timestamp.

### Claim 3 (depends on 1)

The method of claim 1, further comprising, prior to the comparing, altering the local timestamp based on one or more contextual cues.

### Claim 4 (depends on 3)

The method of claim 3, wherein the one or more contextual cues include the first electronic device being assigned a different priority than the one or more additional devices.

### Claim 5 (depends on 4)

The method of claim 4, wherein the first electronic device is assigned the different priority based on the first electronic device being integral with a vehicle.

### Claim 6 (depends on 4)

The method of claim 4, wherein the different priority assigned to the first electronic device is lower than one or more priorities assigned to the one or more additional electronic devices that are integral with a vehicle.

### Claim 7 (depends on 3)

The method of claim 3, wherein the altering comprises adding or subtracting time from the local timestamp.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
subsequent to causing the first electronic device to invoke the automated assistant locally, detecting negative feedback from the user; and
causing the first electronic device to emit another light emission, wherein the another emission causes the one of the additional electronic devices to invoke the automated assistant.

### Claim 9 (depends on 8)

The method of claim 8, further comprising causing the first electronic device to defer to the one of the additional electronic devices.

### Claim 10 (depends on 8)

The method of claim 8, wherein detecting the negative feedback comprises:
detecting a follow-up spoken utterance at the microphone; and
processing the follow-up spoken utterance to determine that the user intended that the automated assistant be invoked on one of the additional electronic devices.

### Claim 11

A first electronic device comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:
detect, at a microphone of the first electronic device, a spoken utterance provided by a user;
detect one or more light emissions emitted by one or more additional electronic devices, wherein each of the one or more light emissions encodes one or more timestamps corresponding to detection of the spoken utterance at a respective electronic device of the one or more additional electronic devices;
extract, from the one or more light emissions emitted by the one or more additional electronic devices, the one or more timestamps;
compare the one or more timestamps to a local timestamp corresponding to detection of the spoken utterance at the first electronic device; and
based on the comparison, cause the first electronic device to either invoke an automated assistant locally or defer to one of the additional electronic devices.

### Claim 12 (depends on 11)

The first electronic device of claim 11, further comprising instructions to cause the first electronic device to emit a light emission that encodes the local timestamp.

### Claim 13 (depends on 11)

The first electronic device of claim 11, further comprising instructions to alter the local timestamp based on one or more contextual cues.

### Claim 14 (depends on 13)

The first electronic device of claim 13, wherein the one or more contextual cues include the first electronic device being assigned a different priority than the one or more additional devices.

### Claim 15 (depends on 14)

The first electronic device of claim 14, wherein the first electronic device is assigned the different priority based on the first electronic device being integral with a vehicle.

### Claim 16 (depends on 14)

The first electronic device of claim 14, wherein the different priority assigned to the first electronic device is lower than one or more priorities assigned to the one or more additional electronic devices that are integral with a vehicle.

### Claim 17 (depends on 13)

The first electronic device of claim 13, wherein the altering comprises adding or subtracting time from the local timestamp.

### Claim 18 (depends on 11)

The first electronic device of claim 11, further comprising instructions to:
subsequent to invocation of the automated assistant locally at the first electronic device, detect negative feedback from the user; and
cause the first electronic device to emit another light emission, wherein the another light emission causes the one of the additional electronic devices to invoke the automated assistant.

### Claim 19 (depends on 18)

The first electronic device of claim 18, further comprising instructions to cause the first electronic device to defer to the one of the additional electronic devices.

### Claim 20

At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors of a first electronic device, cause the one or more processors to:
detect, at a microphone of the first electronic device, a spoken utterance provided by a user;
detect one or more light emissions emitted by one or more additional electronic devices, wherein each of the one or more light emissions encodes one or more timestamps corresponding to detection of the spoken utterance at a respective electronic device of the one or more additional electronic devices;
extract, from the one or more light emissions emitted by the one or more additional electronic devices, the one or more timestamps;
compare the one or more timestamps to a local timestamp corresponding to detection of the spoken utterance at the first electronic device; and
based on the comparison, cause the first electronic device to either invoke an automated assistant locally or defer to one of the additional electronic devices.