# US 2020/0395016 A1 · Voice to voice natural language understanding processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0395016 A1
- Document ID: US20200395016A1
- Status: Published application
- Kind: A1
- Application number: 16/925,045
- Filed: 2020-07-09
- Published: 2020-12-17
- Earliest claimed: 2018-06-13
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Saiprasad Satya Kapila; Antonio Melis; Manikya Pavan Kiran Pothukuchi; Steven Rabuchin; Robert Pulciani; Robert William Serr
- Web page: https://patentgrep.com/patent/US20200395016A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06Q 30/0621: {by configuring or customising goods or services}
- G06Q 30/0635: {replenishment orders; recurring orders}
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 15/1807: {using prosody or stress}
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Techniques for providing a standardized voice user interface (VUI) that enables voice to voice natural language understanding (NLU) processing are described. The standardized VUI may be added to speechlets to enable a customer to interact with a business via server(s) using NLU processing. For example, a first user may initiate a first voice interaction using NLU processing with the server(s) and the system may initiate a second voice interaction using NLU processing between the server(s) and a second user. This enables a customer to initiate a transaction (e.g., request information, place an order, make a reservation, etc.) with the business using the speechlet. Thus, the business may use the speechlet to offer services without requiring additional infrastructure or complicated programming to implement.

## Claims

### Claim at position 1

-20. (canceled)

### Claim 21

A computer-implemented method comprising:
receiving, from a first device, first input audio data corresponding to a first utterance;
performing speech processing using the first input audio data to determine first intent data;
performing processing with regard to the first intent data to determine first output data;
storing the first output data;
after storing the first output data, receiving, from a second device, second input audio data corresponding to a second utterance;
performing speech processing using the second input audio data and the first output data to determine second intent data;
performing processing with regard to the second intent data to determine second output data;
performing speech synthesis using the second output data to determine output audio data responsive to the second utterance; and
sending the output audio data to the second device for output.

### Claim 22 (depends on 21)

The computer-implemented method of claim 21, further comprising:
determining first data is needed to execute an action corresponding to the first intent data;
determining third output data corresponding to a request for the first data; and
sending the third output data to the first device.

### Claim 23 (depends on 22)

The computer-implemented method of claim 22, wherein:
the third output data comprises audio data; and
the method further comprises performing speech synthesis to determine the third output data.

### Claim 24 (depends on 22)

The computer-implemented method of claim 22, further comprising:
receiving third input audio data corresponding to a response to the third output data; and
performing speech processing using the third input audio data to determine the first data,
wherein the first output data is based at least in part on the first data.

### Claim 25 (depends on 21)

The computer-implemented method of claim 21, further comprising:
determining entity data corresponding to the first intent data,
wherein the first output data is based at least in part on the entity data.

### Claim 26 (depends on 21)

The computer-implemented method of claim 21, wherein:
the first device is associated with a first profile; and
the second device is associated with the first profile.

### Claim 27 (depends on 21)

The computer-implemented method of claim 21, wherein:
the first output data is associated with a first profile; and
the method further comprises:
determining first data is needed to execute an action corresponding to the second intent data;
determining third output data corresponding to a request for the first data;
based at least in part on the first profile, determining a recipient device for the third output data; and
sending the third output data to the recipient device.

### Claim 28 (depends on 21)

The computer-implemented method of claim 21, wherein:
the first output data is associated with a first profile; and
the method further comprises:
determining first data is needed to execute an action corresponding to the second intent data;
based at least in part on the first profile, determining third output data corresponding to a request for the first data; and
sending the third output data to the second device.

### Claim 29 (depends on 21)

The computer-implemented method of claim 21, wherein performing speech processing using the second input audio data and the first output data comprises performing natural language understanding processing using at least the first output data.

### Claim 30 (depends on 21)

The computer-implemented method of claim 21, wherein:
the first device corresponds to a first profile;
the first profile corresponds to an enabled speechlet component; and
the second intent data corresponds to the enabled speechlet component.

### Claim 31

A system comprising:
at least one processor; and
at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:
receive, from a first device, first input audio data corresponding to a first utterance;
perform speech processing using the first input audio data to determine first intent data;
perform processing with regard to the first intent data to determine first output data;
store the first output data;
after storing the first output data, receive, from a second device, second input audio data corresponding to a second utterance;
perform speech processing using the second input audio data and the first output data to determine second intent data;
perform processing with regard to the second intent data to determine second output data;
perform speech synthesis using the second output data to determine output audio data responsive to the second utterance; and
send the output audio data to the second device for output.

### Claim 32 (depends on 31)

The system of claim 31, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine first data is needed to execute an action corresponding to the first intent data;
determine third output data corresponding to a request for the first data; and
send the third output data to the first device.

### Claim 33 (depends on 32)

The system of claim 32, wherein:
the third output data comprises audio data; and
the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to perform speech synthesis to determine the third output data.

### Claim 34 (depends on 32)

The system of claim 32, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
receive third input audio data corresponding to a response to the third output data; and
perform speech processing using the third input audio data to determine the first data,
wherein the first output data is based at least in part on the first data.

### Claim 35 (depends on 31)

The system of claim 31, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine entity data corresponding to the first intent data,
wherein the first output data is based at least in part on the entity data.

### Claim 36 (depends on 31)

The system of claim 31, wherein:
the first device is associated with a first profile; and
the second device is associated with the first profile.

### Claim 37 (depends on 31)

The system of claim 31, wherein:
the first output data is associated with a first profile; and
wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine first data is needed to execute an action corresponding to the second intent data;
determine third output data corresponding to a request for the first data;
based at least in part on the first profile, determine a recipient device for the third output data; and
send the third output data to the recipient device.

### Claim 38 (depends on 31)

The system of claim 31, wherein:
the first output data is associated with a first profile; and
wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine first data is needed to execute an action corresponding to the second intent data;
based at least in part on the first profile, determine third output data corresponding to a request for the first data; and
send the third output data to the second device.

### Claim 39 (depends on 31)

The system of claim 31, wherein the instructions that cause the system to perform speech processing using the second input audio data and the first output data comprise instructions that, when executed by the at least one processor, further cause the system to perform natural language understanding processing using at least the first output data.

### Claim 40 (depends on 31)

The system of claim 31, wherein:
the first device corresponds to a first profile;
the first profile corresponds to an enabled speechlet component; and
the second intent data corresponds to the enabled speechlet component.
