# US 2024/0185852 A1 · Multiple skills processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0185852 A1
- Document ID: US20240185852A1
- Status: Published application
- Kind: A1
- Application number: 18/411,516
- Filed: 2024-01-12
- Published: 2024-06-06
- Earliest claimed: 2019-03-06
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Rohin Dabas; Troy Dean Schuring; Rashmi Tonge; Michael James Montgomery; Kevindra Pal Singh; Adam Baran; David Thomas; Nnenna Eleanya Okwara
- Web page: https://patentgrep.com/patent/US20240185852A1

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 15/183: using context dependencies, e.g. language models
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Described herein is a system for enabling a user to perform complex goals using multiple skills/applications of an intelligent assistant device. Skills may register as consumers of an action or providers of an action, and the consumer skills may be configured to invoke provider skills to perform actions. The system receives a request to perform an action from a skill along with some action data. The system validates the action data, selects another skill to perform the action, and forwards the request to the selected skill to perform the action.

## Claims

### Claim at position 1

-20. (canceled)

### Claim 21

A computer-implemented method, comprising:
receiving first data indicating a first component, capable of performing a first action in response to a natural language input, is authorized to request performance of a second action that is performable by at least one other component different than the first component;
after receiving the first data, receiving input data representing a first natural language input;
determining, using the input data, that the first action is to be performed using the first component;
sending, to the first component, a first request to perform the first action in response to the first natural language input;
receiving, from the first component and after sending the first request, a second request for a second component to perform the second action, the second action being determined by the first component;
determining, using the first data, that the first component is authorized to request performance of the second action;
sending, to the second component, a third request to perform the second action; and
determining, based at least in part on performance of the first action and the second action, output data responsive to the first natural language input, the output data comprising a first natural language output.

### Claim 22 (depends on 21)

The computer-implemented method of claim 21, wherein determining the output data comprising the first natural language output is performed by the first component.

### Claim 23 (depends on 21)

The computer-implemented method of claim 21, further comprising:
determining a user profile associated with the input data; and
determining, based at least in part on the user profile, that the second component is permitted to execute with respect to the input data.

### Claim 24 (depends on 21)

The computer-implemented method of claim 21, further comprising:
detecting, by a first device, a user input corresponding to the input data;
executing, by the first device, the first component; and
executing, by the first device, the second component.

### Claim 25 (depends on 21)

The computer-implemented method of claim 21, further comprising:
receiving a user input corresponding to an indication for the first component to be permitted to request performance of the second action; and
based at least in part on the user input, determining the first data.

### Claim 26 (depends on 21)

The computer-implemented method of claim 21, further comprising:
after receiving the input data and prior to sending the third request, determining that second data is required to perform the second action by the second component; and
generating output data requesting the second data from a user device.

### Claim 27 (depends on 21)

The computer-implemented method of claim 21, wherein the second request comprises an application program interface (API) call.

### Claim 28 (depends on 21)

The computer-implemented method of claim 21, further comprising:
determining, by the first component, second data corresponding to execution of the second action; and
sending, from the first component to the second component, the second data.

### Claim 29 (depends on 28)

The computer-implemented method of claim 28, wherein the second data comprises natural language data.

### Claim 30 (depends on 21)

The computer-implemented method of claim 21, further comprising:
determining, by the first component, second data corresponding to execution of the second action by the second component; and
causing a user device to display an output corresponding to the second data.

### Claim 31

A system comprising:
at least one processor; and
at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:
receive first data indicating a first component, capable of performing a first action in response to a natural language input, is authorized to request performance of a second action that is performable by at least one other component different than the first component;
after receipt of the first data, receiving input data representing a first natural language input;
determine, using the input data, that the first action is to be performed using the first component;
send, to the first component, a first request to perform the first action in response to the first natural language input;
receive, from the first component and after sending the first request, a second request for a second component to perform the second action, the second action being determined by the first component;
determine, using the first data, that the first component is authorized to request performance of the second action;
send, to the second component, a third request to perform the second action; and
determine, based at least in part on performance of the first action and the second action, output data responsive to the first natural language input, the output data comprising a first natural language output.

### Claim 32 (depends on 31)

The system of claim 31, wherein determination of the output data comprising the first natural language output is performed by the first component.

### Claim 33 (depends on 31)

The system of claim 31, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine a user profile associated with the input data; and
determine, based at least in part on the user profile, that the second component is permitted to execute with respect to the input data.

### Claim 34 (depends on 31)

The system of claim 31, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
detect, by a first device, a user input corresponding to the input data;
execute, by the first device, the first component; and
execute, by the first device, the second component.

### Claim 35 (depends on 31)

The system of claim 31, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
receive a user input corresponding to an indication for the first component to be permitted to request performance of the second action; and
based at least in part on the user input, determine the first data.

### Claim 36 (depends on 31)

The system of claim 31, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
after receipt of the input data and prior to sending the third request, determine that second data is required to perform the second action by the second component; and
generate output data requesting the second data from a user device.

### Claim 37 (depends on 31)

The system of claim 31, wherein the second request comprises an application program interface (API) call.

### Claim 38 (depends on 31)

The system of claim 31, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determining, by the first component, second data corresponding to execution of the second action; and
sending, from the first component to the second component, the second data.

### Claim 39 (depends on 38)

The system of claim 38, wherein the second data comprises natural language data.

### Claim 40 (depends on 31)

The system of claim 31, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine, by the first component, second data corresponding to execution of the second action by the second component; and
cause a user device to display an output corresponding to the second data.
