# US 12,217,749 B1 · Device targeting for content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,749 B1
- Document ID: US12217749B1
- Status: Granted
- Kind: B1
- Application number: 17/547,664
- Filed: 2021-12-10
- Granted: 2025-02-04
- Earliest claimed: 2021-12-10
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Ratika Anand; Zhen Hua; Trisha Hajela; Evan Victor Chang; Tom Vasella
- Web page: https://patentgrep.com/patent/US12217749B1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Devices and techniques are generally described for targeting of devices. In various examples, a first natural language input comprising a first request to output a response may be received by an input device. A first component may determine first data associated with the input device. A plurality of devices associated with the first data may be determined. First state data describing a state of each device of the plurality of devices may be determined. A first device of the plurality of devices may be determined as a target device for the first request based at least in part on the first state data. The first device may be different from the input device. First instructions may be sent to the first device effective to cause the first device to display the first visual content.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving, from an input device, a first natural language input comprising a first request to display first visual content;
determining that the input device is associated with a first account;
determining, using a natural language understanding component, first intent data representing a semantic interpretation of the first request, the first intent data effective to cause the first visual content to be displayed;
sending, by a target determination component to a device context component, identifier data identifying the first account;
receiving device context data from the device context component, wherein the device context data comprises a list of devices associated with the first account, first data representing a first output device of the list of devices being active, second data indicating that a second output device of the list of devices includes a display, and third data indicating a state of a third output device of the list of devices;
determining, by the target determination component using the first data, the first output device and the second output device among the list of devices that have an active state;
determining, by the target determination component, that the first output device has been more recently active relative to the second output device; and
sending command data to the first output device, the command data causing the first output device to display the first visual content.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
sending, by the device context component, the identifier data to an account component;
receiving, by the device context component from the account component, the list of devices;
sending, by the device context component, data identifying the list of devices in a request for the first data, the second data, and the third data; and
receiving, by the device context component, the first data, the second data, and the third data.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, by a first component, state update data indicating that the first output device has paused playback of second visual content;
sending, by the first component, a request to the target determination component to precompute visual target data;
determining, by the target determination component, that the first output device is a visual target device;
storing data representing the visual target device in non-transitory computer-readable memory;
receiving a second natural language input comprising a second request to resume playback;
retrieving the data representing the visual target device from the non-transitory computer-readable memory; and
sending command data to the first output device, the command data causing the first output device to resume playback of the second visual content.

### Claim 4

A method comprising:
receiving, from an input device, a first natural language input comprising a first request to output a response;
determining, by a first component, first data associated with the input device;
determining a first output device associated with the first data;
determining a second output device associated with the first data;
determining first state data describing a state of the first output device;
determining second state data describing a state of the second output device;
determining the first output device as a target for the first request based at least in part on the first state data, wherein the first output device is different from the input device; and
sending first command data to the first output device, the first command data causing the first output device to display first visual content.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
receiving, by a second component using an event driven architecture, message data indicating that a state of the first output device has changed;
determining, based at least in part on the message data, visual target data identifying the first output device;
storing the visual target data in non-transitory computer-readable memory;
receiving a second natural language input comprising a second request;
determining the visual target data stored in the non-transitory computer-readable memory; and
sending second command data related to the second request to the first output device based at least in part on the visual target data.

### Claim 6 (depends on 4)

The method of claim 4, further comprising:
determining second data among the first state data indicating that the first output device is paired with the input device;
determining third data among the first state data indicating the first output device includes a display; and
determining the first output device as the target for the first request further based at least in part on the second data and the third data.

### Claim 7 (depends on 4)

The method of claim 4, further comprising:
determining second data among the first state data indicating a set of output devices having an applicable state to display the first visual content;
determining third data among the first state data indicating that the first output device is a most recently active output device among the set of output devices; and
determining the first output device as the target for the first request based at least in part on the third data.

### Claim 8 (depends on 4)

The method of claim 4, further comprising:
receiving, from the input device, a second natural language input comprising a second request to interact with first content;
determining, by the first component, the first data associated with the input device;
determining second state data describing a current state of the input device;
determining, using the second state data, that the input device has a state indicating that a plurality of visual content items are displayed by the input device;
determining the input device as a target for the second request based at least in part on the second state data; and
sending second command data to the input device, the second command data causing an interaction with at least one of the plurality of visual content items.

### Claim 9 (depends on 4)

The method of claim 4, further comprising:
detecting a state change of the second output device;
determining target data identifying the second output device based at least in part on the state change;
storing a visual target data in a non-transitory computer-readable memory;
receiving a second natural language input comprising a second request; and
determining the target for the second request by executing device-targeting command data specific to visual output devices, wherein the device-targeting command data specific to visual output devices are used based on the visual target data stored in the non-transitory computer-readable memory.

### Claim 10 (depends on 4)

The method of claim 4, wherein the first data comprises account identifier data, the method further comprising:
receiving, by the first component, the first output device and the second output device as candidate output devices for the first request based at least in part on the account identifier data;
sending data identifying the first output device and the second output device to a device context component to retrieve the first state data and the second state data; and
receiving, from the device context component, the first state data and the second state data.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
generating, by the device context component using the data identifying the first output device and the second output device, a first processing thread to retrieve state data for the first output device;
generating, by the device context component using the data identifying the first output device and the second output device, a second processing thread to retrieve state data for the second output device;
determining that an amount of time since generating the second processing thread has exceeded a timeout value; and
returning the first state data to the first component.

### Claim 12 (depends on 4)

The method of claim 4, further comprising:
receiving, from the input device, a second natural language input comprising a second request to resume playback;
determining that second state data identifying the first output device as the target for the first request is stored in a non-transitory computer-readable memory;
selecting the first output device as the target for the second request based at least in part on the second state data; and
sending second command data to the first output device, the second command data, causing the first output device to resume playback of the first visual content.

### Claim 13

A system comprising:
at least one processor; and
non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to:
receive, from an input device, a first natural language input comprising a first request to output a response;
determine, by a first component, first data associated with the input device;
determine a first output device associated with the first data;
determine a second output device associated with the first data;
determine first state data describing a state of the first output device;
determine second state data describing a state of the second output device;
determine the first output device as a target for the first request based at least in part on the first state data, wherein the first output device is different from the input device; and
send first command data to the first output device, the first command data causing the first output device to display first visual content.

### Claim 14 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
receive, by a second component using an event driven architecture, message data indicating that a state of the first output device has changed;
determine, based at least in part on the message data, visual target data identifying the first output device;
store the visual target data in the non-transitory computer-readable memory;
receive a second natural language input comprising a second request;
determine the visual target data stored in the non-transitory computer-readable memory; and
send second command data related to the second request to the first output device based at least in part on the visual target data.

### Claim 15 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
determine second data among the first state data indicating that the first output device is paired with the input device;
determine third data among the first state data indicating the first output device includes a display; and
determine the first output device as the target for the first request further based at least in part on the second data and the third data.

### Claim 16 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
determine second data among the first state data indicating a set of output devices having an applicable state to display the first visual content;
determine third data among the first state data indicating that the first output device is a most recently active output device among the set of output devices; and
determine the first output device as the target for the first request based at least in part on the third data.

### Claim 17 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
receive, from the input device, a second natural language input comprising a second request to interact with first content;
determine, by the first component, the first data associated with the input device;
determine second state data describing a current state of the input device;
determine, using the second state data, that the input device has a state indicating that a plurality of visual content items are displayed by the input device;
determine the input device as a target for the second request based at least in part on the second state data; and
send second command data to the input device, the second command data causing an interaction with at least one of the plurality of visual content items.

### Claim 18 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
detect a state change of the second output device;
determine target data identifying the second output device based at least in part on the state change;
store a visual target data in the non-transitory computer-readable memory;
receive a second natural language input comprising a second request; and
determine the target for the second request by executing device-targeting command data specific to visual output devices, wherein the device-targeting command data specific to visual devices are used based on the visual target data stored in non-transitory computer-readable memory.

### Claim 19 (depends on 13)

The system of claim 13, wherein the first data comprises account identifier data, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
receive, by the first component, the first output device and the second output device as candidate output devices for the first request based at least in part on the account identifier data;
send data identifying the first output device and the second output device to a device context component to retrieve the first state data and the second state data; and
receive, from the device context component, the first state data and the second state data.

### Claim 20 (depends on 19)

The system of claim 19, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
generate, by the device context component using the data identifying the first output device and the second output device, a first processing thread to retrieve state data for the first output device;
generate, by the device context component using the data identifying the first output device and the second output device, a second processing thread to retrieve state data for the second output device;
determine that an amount of time since generating the second processing thread has exceeded a timeout value; and
return the first state data to the first component.