# US 10,504,520 B1 · Voice-controlled communication requests and responses

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,504,520 B1
- Document ID: US10504520B1
- Status: Granted
- Kind: B1
- Application number: 15/193,874
- Filed: 2016-06-27
- Granted: 2019-12-10
- Earliest claimed: 2016-06-27
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Tapas Kanti Roy; Brian Oliver; Christo Frank Devaraj
- Web page: https://patentgrep.com/patent/US10504520B1

## CPC

- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- H04M 3/42204: {Arrangements at the exchange for service or number selection by voice (at the terminal H04M1/27)}
- G10L 17/22: Interactive procedures; Man-machine interfaces
- H04M 3/4931: {Directory assistance systems}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/225: {Feedback of the input speech}
- H04M 3/00: Automatic or semi-automatic exchanges
- H04M 2201/39: using speech synthesis
- H04M 2201/40: using speech recognition
- H04M 2203/2038: Call context notifications
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

Systems and methods for establishing communication connections using speech, such as establishing calls between speech-controlled devices, are described. A first speech-controlled device receives a communication request in the form of audio and sends audio data corresponding to the captured audio to a server. The server performs speech processing on the audio data to determine a recipient, a subject for the call, and a device associated with the recipient. The server then sends a message indicating the communication request and audio data corresponding to the communication topic to the recipient's speech-controlled device. The recipient device outputs audio to the recipient requesting whether the recipient accepts the communication request. The recipient audibly refuses or accepts the communication request, and the recipient's speech-controlled device sends an indication of the recipient's audible decision to the server. If the recipient accepted the communication request, the server causes a communication connection be established between the two speech-controlled devices.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving, from a first device, first audio data corresponding to a first spoken user input;
performing speech processing on the first audio data to determine the first spoken user input corresponds to a request to establish a two-way communication via which second audio data is to be sent from the first device to another device;
identifying a recipient name within the first spoken user input;
determining user profile data associated with the first audio data, the user profile data including a contact list;
determining, in the contact list, that the recipient name is associated with a second device indicator;
determining a topic representing the second audio data;
generating third audio data representing the request and the topic;
sending, to a second device corresponding to the second device indicator, the third audio data;
receiving, from the second device, fourth audio data corresponding to a second spoken user input;
performing speech processing on the fourth audio data to determine the second spoken user input corresponds to an acceptance of the request; and
establishing the two-way communication between the first device and the second device.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining, in the contact list, that the recipient name is associated with a third device indicator;
sending, to the second device, a first ringing instruction;
sending, to a third device corresponding to the third device indicator, a second ringing instruction; and
after sending the first and second ringing instructions, receiving the fourth audio data.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein determining the topic comprises:
determining second user profile data associated with the recipient name;
determining a first device indicator, corresponding to the first device, is missing from a list of devices represented in the second user profile data;
based at least in part on determining the first device indicator is missing from the list of devices, generating fifth audio data corresponding to a second request for at least one topic;
sending, to the first device, the fifth audio data; and
receiving, from the first device, sixth audio data corresponding to a third spoken user input, the third spoken user input corresponding to the topic.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein determining the topic comprises:
determining the first spoken user input includes a first topic identifier;
accessing a table associated with the first device, the table including topic identifiers, each of the topic identifiers being associated with different topic language; and
identifying, in the table, first topic language associated with the first topic identifier.

### Claim 5

A computing system, comprising:
at least one processor; and
at least one memory comprising instructions that, when executed by the at least one processor, cause the computing system to:
receive, from a first device, first audio data corresponding to a spoken user input;
perform speech processing on the first audio data to determine the spoken user input corresponds to a request to cause second audio data to be sent from the first device to another device;
determine a recipient name represented in the spoken user input;
determine a topic representing the second audio data;
determine user profile data associated with the first audio data;
determine a contact list associated with the user profile data;
determine, using the contact list, that the recipient name is associated with a second device;
send, to the second device, first data representing the request; and
send, to the second device, second data representing the topic.

### Claim 6 (depends on 5)

The computing system of claim 5, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:
receive, from the second device, an indication of acceptance of the request; and
enable the second audio data to be sent from the first device to the second device.

### Claim 7 (depends on 5)

The computing system of claim 5, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:
receive, from the second device, third audio data corresponding to a second spoken user input;
perform speech processing on the third audio data to determine the second spoken user input corresponds to an acceptance of the request; and
enable the second audio data to be sent from the first device to the second device.

### Claim 8 (depends on 5)

The computing system of claim 5, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:
prior to determining the topic:
identify a plurality of pre-stored topics;
determine the spoken user input does not represent at least one of the plurality of pre-stored topics;
send, to the first device, third data requesting at least one topic; and
receive, from the first device, third audio data corresponding to a second spoken user input representing to the topic.

### Claim 9 (depends on 5)

The computing system of claim 5, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:
determine second user profile data associated with the recipient name;
determine the second user profile data includes a second device indicator corresponding to the second device;
determine the second user profile data includes a third device indicator corresponding to a third device;
send, to the second device, a first ringing instruction;
send, to the third device, a second ringing instruction;
receive, from the second device, an indication representing acceptance of the request; and
enable the second audio data to be sent from the first device to the second device.

### Claim 10 (depends on 5)

The computing system of claim 5, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:
receive, from the first device, third audio data corresponding to a second spoken user input;
perform speech processing on the third audio data to determine the second spoken user input corresponds to a second request to cause fourth audio data to be sent from the first device to another device;
determine a second recipient name represented in the second spoken user input;
determine a third device associated with the second recipient name;
send, to the third device, third data representing the second request;
receive, from the third device, fifth audio data corresponding to a third spoken user input;
perform speech processing on the fifth audio data to determine the third spoken user input corresponds to a declination of the second request; and
send, to the first device, fourth data representing the second request was declined.

### Claim 11 (depends on 5)

The computing system of claim 5, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:
receive, from the first device, third audio data corresponding to a second spoken user input;
perform speech processing on the third audio data to determine the second spoken user input corresponds to a second request to cause fourth audio data to be sent from the first device to another device;
determine a second recipient name represented in the second spoken user input;
determine a third device associated with the second recipient name;
send, to the third device, third data representing the second request;
receive, from the third device, fifth audio data corresponding to a third spoken user input;
perform speech processing on the fifth audio data to determine the third spoken user input includes reschedule information; and
send, to a calendar component associated with the first device, fourth data representing the reschedule information.

### Claim 12

A computer-implemented method, comprising:
receiving, from a first device, first audio data corresponding to a spoken user input;
performing speech processing on the first audio data to determine the spoken user input corresponds to a request to cause second audio data to be sent from the first device to another device;
determining a recipient name represented in the spoken user input;
identifying a keyword in the spoken user input;
determining a topic representing the second audio data, the topic corresponding to a portion of the spoken user input following the keyword;
determining a second device associated with the recipient name;
sending, to the second device, first data representing the request; and
sending, to the second device, second data representing the topic.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, further comprising:
receiving, from the second device, an indication of acceptance of the request; and
enabling the second audio data to be sent from the first device to the second device.

### Claim 14 (depends on 12)

The computer-implemented method of claim 12, further comprising:
receiving, from the second device, third audio data corresponding to a second spoken user input;
performing speech processing on the third audio data to determine the second spoken user input corresponds to an acceptance of the request; and
enabling the second audio data to be sent from the first device to the second device.

### Claim 15 (depends on 12)

The computer-implemented method of claim 12, further comprising:
determining user profile data associated with the first audio data;
determining a contact list associated with the user profile data;
determining, using the contact list, that the recipient name is associated with a second device indicator corresponding to the second device; and
determining the second device based at least in part on determining the recipient name is associated with the second device indicator in the contact list.

### Claim 16 (depends on 12)

The computer-implemented method of claim 12, further comprising:
determining user profile data associated with the recipient name;
determining the user profile data includes a second device indicator corresponding to the second device;
determining the user profile data includes a third device indicator corresponding to a third device;
sending, to the second device, a first ringing instruction;
sending, to the third device, a second ringing instruction;
receiving, from the second device, an indication representing acceptance of the request; and
enabling the second audio data to be sent from the first device to the second device.

### Claim 17 (depends on 12)

The computer-implemented method of claim 12, further comprising:
receiving, from the first device, third audio data corresponding to a second spoken user input;
performing speech processing on the third audio data to determine the second spoken user input corresponds to a second request to cause fourth audio data to be sent from the first device to another device;
determining a second recipient name represented in the second spoken user input;
determining a third device associated with the second recipient name;
sending, to the third device, third data representing the second request;
receiving, from the third device, fifth audio data corresponding to a third spoken user input;
performing speech processing on the fifth audio data to determine the third spoken user input corresponds to a declination of the second request; and
sending, to the first device, fourth data representing the second request was declined.

### Claim 18 (depends on 12)

The computer-implemented method of claim 12, further comprising:
receiving, from the first device, third audio data corresponding to a second spoken user input;
performing speech processing on the third audio data to determine the second spoken user input corresponds to a second request to cause fourth audio data to be sent from the first device to another device;
determining a second recipient name represented in the second spoken user input;
determining a third device associated with the second recipient name;
sending, to the third device, third data representing the second request;
receiving, from the third device, fifth audio data corresponding to a third spoken user input;
performing speech processing on the fifth audio data to determine the third spoken user input includes reschedule information; and
sending, to a calendar component associated with the first device, fourth data representing the reschedule information.

### Claim 19 (depends on 12)

The computer-implemented method of claim 12, wherein the recipient name corresponds to a name of an individual in a contact list associated with the first device.