# US 12,008,985 B2 · Natural language processing of declarative statements

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,985 B2
- Document ID: US12008985B2
- Status: Granted
- Kind: B2
- Application number: 16/907,680
- Filed: 2020-06-22
- Published: 2021-12-23
- Granted: 2024-06-11
- Earliest claimed: 2020-06-22
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Qiaozi Gao; Divyanshu Brijmohan Verma; Govindarajan Sundaram Thattai; Qing Ping; Joel Joseph Chengottusseriyil; Ivan Vitomir Stojanovic; Feiyang Niu; Gokhan Tur; Charles J Allen
- Web page: https://patentgrep.com/patent/US12008985B2

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G06F 40/30: Semantic analysis
- G06F 40/35: Discourse or dialogue representation
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 13/08: Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/225: {Feedback of the input speech}
- G10L 2015/227: {of the speaker; Human-factor methodology}
- G10L 2015/228: {of application context}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Devices and techniques are generally described for learning personalized responses to declarative natural language inputs. In various examples, a first natural language input may be received. The first natural language input may correspond to intent data corresponding to a declarative user input. In some examples, a dialog session may be initiated with the first user. An action intended by the first user for the first natural language input may be determined based at least in part on the dialog session. In various examples, first data representing the action may be stored in association with second data representing a state described by at least a portion of the first natural language input.

## Claims

### Claim 1

A method comprising:
receiving, by a natural language processing system at a first time, first natural language input data representing a declarative statement from a device associated with a user profile;
determining, by the natural language processing system, that the first natural language input data corresponds to a declarative intent;
determining first state data indicated by the declarative intent, the first state data representing an environmental condition associated with the first natural language input data;
determining a predicted action corresponding to the first state data;
generating first output data comprising a prompt to inquire if the predicted action is intended;
receiving second natural language input data, wherein the second natural language input data represents an action requested by a user that is different from the predicted action;
storing, in a non-transitory computer-readable memory, data associated with the user profile and the data representing the action in association with the first state data;
receiving, by the natural language processing system at a second time after the first time, third natural language input data;
determining, by the natural language processing system, that the third natural language input data corresponds to the declarative intent;
determining that the third natural language input data corresponds to the first state data;
determining that the first state data is associated with the data representing the action in the non-transitory computer-readable memory; and
executing the data representing the action to generate a first output.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving, by the natural language processing system, fourth natural language input data;
determining, by the natural language processing system, that the fourth natural language input data corresponds to the declarative intent;
determining second state data for the fourth natural language input data;
determining a second predicted action corresponding to the second state data;
generating second output data comprising a second prompt to inquire if the second predicted action is intended;
receiving, by the natural language processing system, fifth natural language input data;
determining from the fifth natural language input data that the second state data corresponds to the first state data;
determining that the first state data is associated with the data representing the action in the non-transitory computer-readable memory; and
executing the data representing the action to generate the first output.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving, by the natural language processing system, fourth natural language input data representing a second declarative statement associated with a second user profile;
determining, by the natural language processing system, that the fourth natural language input data corresponds to the declarative intent for declarative utterances;
determining that the fourth natural language input data corresponds to the first state data;
determining that the first state data is associated with second data associated with the second user profile, the second data representing a different action; and
executing the second data representing the different action to generate a second output different from the first output.

### Claim 4

A method comprising:
receiving first input data representing a first natural language declarative statement;
determining that the first input data corresponds to a declarative intent;
determining, based at least in part on the first input data corresponding to the declarative intent, first data representing a first state described by at least a portion of the first input data;
generating, by a first machine learning model, second data based at least in part on the first data, wherein the second data represents a predicted action for the first natural language declarative statement;
generating output data representing a request for clarification based at least in part on the predicted action;
receiving second input data representing a response to the request for clarification;
determining an action intended to be taken in response to the first input data based at least in part on the second input data and the first input data;
storing the first data representing the first state in association with third data representing the action in a first data structure;
receiving third input data representing a second natural language declarative statement different from the first natural language declarative statement;
determining that the third input data corresponds to the declarative intent;
determining fourth data representing a second state described by at least a portion of the second natural language declarative statement;
determining that the second state corresponds to the first state;
receiving, from the first data structure, the third data representing the action; and
generating a natural language prompt as output data, the natural language prompt representing a prompt for executing the action.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
receiving fourth input data, the fourth input data requesting a different action; and
storing fifth data representing the different action in association with the first data based at least in part on the fourth input data.

### Claim 6 (depends on 4)

The method of claim 4, wherein the first state comprises a first condition described by at least a portion of the first natural language declarative statement, the method further comprising:
generating second output data representing a prompt inquiring whether the predicted action is to be executed; and
receiving fourth input data indicating that the action is to be executed, wherein the action is different from the predicted action, wherein storing the third data representing the action is performed based at least in part on the fourth input data.

### Claim 7 (depends on 4)

The method of claim 4, further comprising:
receiving fifth data describing a context of the first input data;
determining a first domain associated with the first input data based at least in part on the fourth fifth data; and
determining the declarative intent from among intents associated with the first domain.

### Claim 8 (depends on 4)

The method of claim 4, further comprising:
receiving metadata indicating an account associated with the first input data;
determining at least one address in memory associated with the account; and
storing fifth data representing the action in association with sixth data representing the first natural language declarative statement at the at least one address in memory.

### Claim 9 (depends on 4)

The method of claim 4, wherein the first state comprises a first condition of an environment described by at least a portion of the first natural language declarative statement, the method further comprising:
receiving fourth input data;
determining fifth data representing a third state described by at least a portion of the fourth input data, wherein the third state describes a second condition of the environment;
determining that the first state corresponds to the third state; and
storing the fifth data in association with the action and the first data.

### Claim 10 (depends on 4)

The method of claim 4, further comprising:
receiving fifth data describing a context of the first input data;
determining, by the first machine learning model using the first data and the fifth data, the second data representing the predicted action; and
determining, by a second machine learning model using the first data and the second data, the third data representing the action.

### Claim 11

A system comprising:
at least one processor; and
at least one non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to program the at least one processor to:
receive first input data representing a first natural language declarative statement;
determine that the first input data corresponds to a declarative intent;
determine, based at least in part on the first input data corresponding to the declarative intent, first data representing a first state described by at least a portion of the first input data;
generate, by a first machine learning model, second data based at least in part on the first data, wherein the second data represents a predicted action for the first natural language declarative statement;
generate output data representing a request for clarification based at least in part on the predicted action;
receive second input data representing a response to the request for clarification;
determine an action intended to be taken in response to the first input data based at least in part on the second input data and the first input data;
store the first data representing the first state in association with third data representing the action in a first data structure;
receive third input data representing a second natural language declarative statement different from the first natural language declarative statement;
determine that the third input data corresponds to the declarative intent;
determine fourth data representing a second state described by at least a portion of the second natural language declarative statement;
determine that the second state corresponds to the first state;
receive, from the first data structure, the third data representing the action; and
generate a natural language prompt as output data, the natural language prompt representing a prompt for executing the action.

### Claim 12 (depends on 11)

The system of claim 11, the at least one non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
receive fourth input data, the fourth input data requesting a different action; and
store fifth data representing the different action in association with the first data based at least in part on the fourth input data.

### Claim 13 (depends on 11)

The system of claim 11, wherein the first state comprises a first condition described by at least a portion of the first natural language declarative statement, the at least one non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
generate second output data representing a prompt inquiring whether the predicted action is to be executed; and
receive fourth input data indicating that the action is to be executed, wherein the action is different from the predicted action, wherein storing the third data representing the action is performed based at least in part on the fourth input data.

### Claim 14 (depends on 11)

The system of claim 11, the at least one non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
receive fifth data describing a context of the first input data;
determine a first domain associated with the first input data based at least in part on the fifth data; and
determine the declarative intent from among intents associated with the first domain.

### Claim 15 (depends on 11)

The system of claim 11, the at least one non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
receive metadata indicating an account associated with the first input data;
determine at least one address in memory associated with the account; and
store fifth data representing the action in association with sixth data representing the first natural language declarative statement at the at least one address in memory.

### Claim 16 (depends on 11)

The system of claim 11, the at least one non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
receive fifth data describing a context of the first input data;
determine, by the first machine learning model using the first data and the fifth data, the second data representing the predicted action; and
determine, by a second machine learning model using the first data and the second data, the third data representing the action.