# US 2021/0241760 A1 · Speech-processing system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241760 A1
- Document ID: US20210241760A1
- Status: Published application
- Kind: A1
- Application number: 16/781,773
- Filed: 2020-02-04
- Published: 2021-08-05
- Earliest claimed: 2020-02-04
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Timothy Whalin; Catherine Michelle Loo; Calvin Phuong Nguyen
- Web page: https://patentgrep.com/patent/US20210241760A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 40/30: Semantic analysis
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G06F 40/279: Recognition of textual entities
- G06F 40/295: Named entity recognition
- G06F 40/35: Discourse or dialogue representation
- G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system may include first and second speech-processing systems. The first speech-processing system may process received audio data and determine that a command represented therein is associated with a second speech-processing system. The first speech-processing system may send command data to the second speech-processing system and receive response data in return. The first speech-processing system may then process the response data to determine second response data that includes an indication of the second speech-processing system and cause output of audio corresponding to the second response data.

## Claims

### Claim 1

A method for processing data representing a spoken command, the method comprising:
receiving, from a voice-controlled device, first audio data representing an utterance;
determining that the first audio data includes:
a first portion of audio data representing a wakeword corresponding to a first speech-processing system having a first style of synthesized speech, and
a second portion of audio data representing a command;
determining a domain associated with the command;
determining that the domain is associated with a second speech-processing system having a second style of synthesized speech;
sending, from the first speech-processing system to the second speech-processing system, command data including the command;
receiving, from the second speech-processing system, a first response to the command, the first response including a representation of first natural language corresponding to the second style of synthesized speech;
determining, using the first speech-processing system and the first response, a second response to the command, the second response including a representation of second natural language corresponding to the first style of synthesized speech; and
sending second audio data to the voice-controlled device corresponding to the second response.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining that the command corresponds to an action set to occur at a later time; and
sending, from the second speech-processing system to the voice-controlled device, a second command to execute the action,
wherein the second audio data includes a representation of a name of the second speech-processing system.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving, from the voice-controlled device, third audio data representing a second utterance;
determining that the third audio data corresponds to a second command;
determining that the second command corresponds to a user device proximate a user; and
sending, to the user device via an application programming interface, an instruction corresponding to the second command.

### Claim 4

A computer-implemented method comprising:
receiving input data corresponding to a command;
determining, using a first natural-understanding system, that the command is associated with a second natural-understanding system;
sending, from the first natural-understanding system to the second natural-understanding system, first data corresponding to the command;
receiving, from the second natural-understanding system, second data corresponding to a first response to the command, the second data including a first indication of the second natural-understanding system;
determining, using the first natural-understanding system and the second data, third data corresponding to a second response to the command, the third data including a second indication of the first natural-understanding system; and
causing output corresponding to the third data.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, further comprising:
prior to causing the output, determining that the second natural-understanding system is associated with a request for permission;
determining to cause second output corresponding to the request for permission before sending the first data;
receiving fourth input data corresponding to the request for permission,
wherein the third data further includes a representation of the request for permission.

### Claim 6 (depends on 4)

The computer-implemented method of claim 4, further comprising:
determining that the command corresponds to a second output of the second natural-understanding system, the second output occurring after the output,
wherein the third data includes a representation of a name of the second natural-understanding system.

### Claim 7 (depends on 4)

The computer-implemented method of claim 4, further comprising:
determining that receiving the input data is associated with a first device; and
determining that causing the output is associated with a second device,
wherein the third data includes a representation of a name of the second natural-understanding system.

### Claim 8 (depends on 4)

The computer-implemented method of claim 4, further comprising:
determining that receiving the input data is associated with a first device; and
determining a user account associated with the first device;
determining that the user account includes a third indication of the second natural-understanding system,
wherein the third data includes a representation of a name of the second natural-understanding system.

### Claim 9 (depends on 4)

The computer-implemented method of claim 4, further comprising:
determining that the input data includes a representation of a wakeword associated with the first natural-understanding system.

### Claim 10 (depends on 4)

The computer-implemented method of claim 4, wherein determining that the command is associated with the second natural-understanding system comprises:
determining a domain corresponding to the input data; and
determining that the second natural-understanding system corresponds to the domain.

### Claim 11 (depends on 4)

The computer-implemented method of claim 4, further comprising:
receiving second audio data corresponding to a second command;
determining that the second command is associated with the second natural-understanding system; and
sending, to the second natural-understanding system, a third command to process third audio data.

### Claim 12 (depends on 4)

The computer-implemented method of claim 4, further comprising:
determining that the second data lacks first information;
sending, from the first natural-understanding system to the second natural-understanding system, fourth data corresponding to a request for the first information; and
receiving, from the second natural-understanding system, fifth data corresponding to the first information.

### Claim 13

A computing device comprising:
at least one processor; and
at least one memory including instructions that, when executed by the at least one processor, cause the computing device to:
receive input data corresponding to a command;
determine, using a first natural-understanding system, that the command is associated with a second natural-understanding system;
send, from the first natural-understanding system to the second natural-understanding system, first data corresponding to the command;
receive, from the second natural-understanding system, second data corresponding to a first response to the command, the second data including a first indication of the second natural-understanding system;
determine, using the first natural-understanding system and the second data, third data corresponding to a second response to the command, the third data including a second indication of the first natural-understanding system; and
cause output corresponding to the third data.

### Claim 14 (depends on 13)

The computing device of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the computing device to:
prior to causing the output, determine that the second natural-understanding system is associated with a request for permission,
determine to cause second output corresponding to the request for permission before sending the first data;
receive fourth input data corresponding to the request for permission,
wherein the third data further includes a representation of the request for permission.

### Claim 15 (depends on 13)

The computing device of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the computing device to:
determine that the command corresponds to a second output of the second natural-understanding system, the second output occurring after the output,
wherein the third data includes a representation of a name of the second natural-understanding system.

### Claim 16 (depends on 13)

The computing device of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the computing device to:
determine that receiving the input data is associated with a first device; and
determine that causing the output is associated with a second device,
wherein the third data includes a representation of a name of the second natural-understanding system.

### Claim 17 (depends on 13)

The computing device of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the computing device to:
determine that receiving the input data is associated with a first device; and
determine a user account associated with the first device;
determine that the user account includes a third indication of the second natural-understanding system,
wherein the third data includes a representation of a name of the second natural-understanding system.

### Claim 18 (depends on 13)

The computing device of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the computing device to:
determine that the input data includes a representation of a wakeword associated with the first natural-understanding system.

### Claim 19 (depends on 13)

The computing device of claim 13, wherein the at least one memory further includes instructions to determine that the command is associated with the second natural-understanding system and that, when executed by the at least one processor, further cause the computing device to:
determine a domain corresponding to the input data; and
determine that the second natural-understanding system corresponds to the domain.

### Claim 20 (depends on 13)

The computing device of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the computing device to:
determine that the second data lacks first information;
send, from the first natural-understanding system to the second natural-understanding system, fourth data corresponding to a request for the first information; and
receive, from the second natural-understanding system, fifth data corresponding to the first information.
