# US 12,010,387 B1 · Content-based voice targeting of devices using slot and task data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,010,387 B1
- Document ID: US12010387B1
- Status: Granted
- Kind: B1
- Application number: 17/488,671
- Filed: 2021-09-29
- Granted: 2024-06-11
- Earliest claimed: 2021-09-29
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Genna Michelle Krecicki Moderhack; Tom Vessella; Clare Elizabeth Veladanda; Monty Eich; Ratika Anand
- Web page: https://patentgrep.com/patent/US12010387B1

## CPC

- H04N 21/47217: {for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312)}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 40/295: Named entity recognition
- G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B 27/11: by using information not detectable on the record carrier
- G11B 27/3036: {Time code signal}
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/439: Processing of audio elementary streams
- G06F 16/24578: {using ranking}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- H04L 12/2803: {Home automation networks}
- H04M 1/05: specially adapted for use on head, throat or breast
- H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)}
- H04R 27/00: Public address systems (circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; {monitoring or testing arrangements for public address systems H04R29/007}; amplifiers H03F)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Devices and techniques are generally described for voice targeting of devices using content type. In various examples, a first natural language input comprising a first request to perform a first action related to first content may be received. A natural language understanding component may be used to determine first intent data representing a semantic interpretation of the first request, where the first intent data is effective to cause a first skill to perform the first action. A named entity recognition component may be used to determine first slot data included in the first request. The first slot data may identify the first content. First task data associated with the first content may be determined. The first task data may associate the first content with playback on a first device. First instructions effective to cause the first device to perform the first action may be sent to the first device.

## Claims

### Claim 1

A method comprising:
receiving a first natural language input comprising a first request to pause playback of a first song;
determining, using a natural language understanding component, first intent data representing a semantic interpretation of the first request, the first intent data effective to cause a first skill associated with a first streaming service to pause music playback;
determining, using a named entity recognition component, first slot data associated with the first request, the first slot data identifying a title of the first song;
identifying first task data stored in non-transitory computer-readable memory, wherein the first task data associates metadata describing the first song with first device identifier data identifying a first device on which audio of the first song is being output, the metadata including the title of the first song, and wherein the first task data is stored in the non-transitory computer-readable memory prior to receiving the first natural language input;
determining that the first slot data identifying the title of the first song corresponds to the title of the first song included in the metadata of the first task data;
determining the first device identifier data from the first task data; and
sending first instructions to pause playback of the first song to the first device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving a second natural language input comprising a second request to initiate playback of the first song on a second device;
determining, using the natural language understanding component, second intent data representing a semantic interpretation of the second request, the second intent data effective to cause the first skill associated with the first streaming service to initiate music playback;
determining, using the named entity recognition component, second slot data associated with the second request, the second slot data comprising the title of the first song;
determining, using the named entity recognition component, third slot data associated with the second request, the third slot data comprising a name of the second device;
generating second task data by modifying the first task data to associate the title of the first song with the name of the second device and by deleting the name of the first device; and
sending second instructions to initiate playback of the first song to the second device.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving a second natural language input comprising a second request to initiate playback of the first song on the first device, wherein the second request is received prior to the first request;
determining, using the natural language understanding component, second intent data representing a semantic interpretation of the second request, the second intent data effective to cause the first skill to initiate music playback;
determining, using the named entity recognition component, second slot data associated with the second request, the second slot data comprising the title of the first song;
determining, using the named entity recognition component, second slot data associated with the second request, the second slot data comprising a name of the first device;
and
sending second instructions to initiate playback of the first song to the first device, wherein the second instructions are sent prior to receiving the first request.

### Claim 4

A method comprising:
receiving a first natural language input comprising a first request to perform a first action related to first content;
determining, using a natural language understanding component, first intent data representing a semantic interpretation of the first request, the first intent data effective to cause a first skill to perform the first action;
determining, using a named entity recognition component, first slot data included in the first request, the first slot data identifying the first content;
identifying first task data associated with the first content, the first task data associating the first content with playback on a first device, and wherein the first task data is stored in non-transitory computer-readable memory prior to receiving the first natural language input; and
sending first instructions to the first device, the first instructions effective to cause the first device to perform the first action.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
receiving a second request to initiate playback of the first content by a second device;
generating second task data by modifying the first task data to associate the first content with playback on the second device and by deleting a portion of the first task data that associates the first content with playback on the first device;
sending second instructions to the second device, the second instructions effective to cause the second device to initiate playback of the first content on the second device; and
sending third instructions to the first device, the third instructions effective to cause the first device to stop playback of the first content.

### Claim 6 (depends on 4)

The method of claim 4, further comprising:
receiving a second request to initiate synchronized playback of the first content by the first device and by a second device;
generating second task data by modifying the first task data to associate the first content with playback on the first device and playback on the second device; and
sending second instructions to the second device, the second instructions effective to cause the second device to initiate playback of the first content on the second device, wherein the playback on the second device is synchronized with playback of the first content by the first device;
receiving a second natural language input comprising a third request to perform a second action related to the first content;
determining, using the natural language understanding component, second intent data representing a semantic interpretation of the third request, the second intent data effective to cause the first skill to perform the second action;
determining, using the named entity recognition component, the first slot data included in the third request, the first slot data identifying the first content;
determining, using the first slot data, that the second task data associates the first content with playback on the first device and on the second device;
sending third instructions to the first device, the third instructions effective to cause the first device to perform the second action; and
sending fourth instructions to the second device, the fourth instructions effective to cause the second device to perform the second action.

### Claim 7 (depends on 4)

The method of claim 4, further comprising:
determining a presence of a user in a space associated with a second device;
determining, using the first task data, the first content being played back on the first device;
sending second instructions to the second device, the second instructions effective to cause the second device to initiate playback of the first content; and
sending third instructions to the first device, the third instructions effective to cause the first device to stop playback of the first content.

### Claim 8 (depends on 4)

The method of claim 4, further comprising:
receiving a second natural language input comprising a second request to perform a second action related to the first content, wherein the second request identifies a name of the first skill providing the first content;
determining, using the natural language understanding component, second intent data representing a semantic interpretation of the second request, the second intent data effective to cause the first skill to perform the second action;
determining, using the named entity recognition component, second slot data included in the second request, the second slot data identifying the name of the first skill;
determining that the first task data associates the name of the first skill with the first content and with the playback on the first device; and
sending second instructions to the first device, the second instructions effective to cause the first device to perform the second action.

### Claim 9 (depends on 4)

The method of claim 4, further comprising:
determining metadata describing one or more aspects of the first content; and
generating the first task data by associating the metadata with device identifier data identifying the first device in a data structure.

### Claim 10 (depends on 4)

The method of claim 4, further comprising:
receiving a second natural language input comprising a second request to perform a second action related to the first content, wherein the second request identifies an aspect of the first content;
determining, using a natural language understanding component, second intent data representing a semantic interpretation of the second request, the second intent data effective to cause the first skill to perform the second action;
determining, using the named entity recognition component, second slot data included in the second request, the second slot data identifying the aspect of the first content;
determining that the first task data associates the aspect of the first content with the playback on the first device; and
sending second instructions to the first device, the second instructions effective to cause the first device to perform the second action.

### Claim 11 (depends on 4)

The method of claim 4, further comprising:
receiving a second natural language input comprising a second request to perform a second action related to the first content, wherein the second request identifies a type of content pertaining to the first content;
determining, using a natural language understanding component, second intent data representing a semantic interpretation of the second request, the second intent data effective to cause the first skill to perform the second action;
determining, using the named entity recognition component, second slot data included in the second request, the second slot data identifying the type of content; and
generating second task data by modifying the first task data to associate the second slot data identifying the type of content with playback on the first device.

### Claim 12 (depends on 4)

The method of claim 4, further comprising:
receiving a second request to initiate synchronized playback of the first content by the first device and by a second device;
generating second task data by modifying the first task data to associate the first content with playback on the first device and playback on the second device;
sending second instructions to the second device, the second instructions effective to cause the second device to initiate playback of the first content on the second device, wherein the playback on the second device is synchronized with playback of the first content by the first device;
receiving a third request to initiate playback of second content, wherein the third request specifies the first device for playback of the second content;
generating third task data by modifying the first task data by deleting an association between the first device and the first content;
generating fourth task data that associates the second content with playback on the first device; and
sending third instructions to the first device, the second instructions effective to cause the first device to initiate playback of the second content on the first device.

### Claim 13

A system comprising:
at least one processor; and
non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to:
receive a first natural language input comprising a first request to perform a first action related to first content;
determine, using a natural language understanding component, first intent data representing a semantic interpretation of the first request, the first intent data effective to cause a first skill to perform the first action;
determine, using a named entity recognition component, first slot data included in the first request, the first slot data identifying the first content;
identify first task data associated with the first content, the first task data associating the first content with playback on a first device, and wherein the first task data is stored in the non-transitory computer-readable memory prior to receiving the first natural language input; and
send first instructions to the first device, the first instructions effective to cause the first device to perform the first action.

### Claim 14 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
receive a second request to initiate playback of the first content by a second device;
generate second task data by modifying the first task data to associate the first content with playback on the second device and by deleting a portion of the first task data that associates the first content with playback on the first device;
send second instructions to the second device, the second instructions effective to cause the second device to initiate playback of the first content on the second device; and
sending third instructions to the first device, the third instructions effective to cause the first device to stop playback of the first content.

### Claim 15 (depends on 14)

The system of claim 14, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
receive a second request to initiate synchronized playback of the first content by the first device and by a second device;
generate second task data by modifying the first task data to associate the first content with playback on the first device and playback on the second device; and
send second instructions to the second device, the second instructions effective to cause the second device to initiate playback of the first content on the second device, wherein the playback on the second device is synchronized with playback of the first content by the first device;
receive a second natural language input comprising a third request to perform a second action related to the first content;
determine, using the natural language understanding component, second intent data representing a semantic interpretation of the third request, the second intent data effective to cause the first skill to perform the second action;
determine, using the named entity recognition component, the first slot data included in the third request, the first slot data identifying the first content;
determine, using the first slot data, that the second task data associates the first content with playback on the first device and on the second device;
send third instructions to the first device, the third instructions effective to cause the first device to perform the second action; and
send fourth instructions to the second device, the fourth instructions effective to cause the second device to perform the second action.

### Claim 16 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
determine a presence of a user in a space associated with a second device;
determine, using the first task data, the first content being played back on the first device;
send second instructions to the second device, the second instructions effective to cause the second device to initiate playback of the first content; and
send third instructions to the first device, the third instructions effective to cause the first device to stop playback of the first content.

### Claim 17 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
receive a second natural language input comprising a second request to perform a second action related to the first content, wherein the second request identifies a name of the first skill providing the first content;
determine, using a natural language understanding component, second intent data representing a semantic interpretation of the second request, the second intent data effective to cause the first skill to perform the second action;
determine, using the named entity recognition component, second slot data included in the second request, the second slot data identifying the name of the first skill;
determine that the first task data associates the name of the first skill with the first content and with the playback on the first device; and
send second instructions to the first device, the second instructions effective to cause the first device to perform the second action.

### Claim 18 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
determine metadata describing one or more aspects of the first content; and
generate the first task data by associating the metadata with device identifier data identifying the first device in a data structure.

### Claim 19 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
receive a second natural language input comprising a second request to perform a second action related to the first content, wherein the second request identifies an aspect of the first content;
determine, using a natural language understanding component, second intent data representing a semantic interpretation of the second request, the second intent data effective to cause the first skill to perform the second action;
determine, using the named entity recognition component, second slot data included in the second request, the second slot data identifying the aspect of the first content;
determine that the first task data associates the aspect of the first content with the playback on the first device; and
send second instructions to the first device, the second instructions effective to cause the first device to perform the second action.

### Claim 20 (depends on 13)

The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to:
receive a second natural language input comprising a second request to perform a second action related to the first content, wherein the second request identifies a type of content pertaining to the first content;
determine, using a natural language understanding component, second intent data representing a semantic interpretation of the second request, the second intent data effective to cause the first skill to perform the second action;
determine, using the named entity recognition component, second slot data included in the second request, the second slot data identifying the type of content; and
generate second task data by modifying the first task data to associate the second slot data identifying the type of content with playback on the first device.