US 10,504,513 B1Granted

Natural language understanding with affiliated devices

AssigneeAmazon Technologies, Inc.

InventorsTimothy Thomas Gray, Michal Grzegorz Kurpanik, Jenny Toi Wah Lam, Sarveshwar Nigam, Shirin Saleem, Jonhenry A. Righter, Jeremy Richard Hill, Kavya Ravikumar, Joe Virgil Fernandez, Kynan Dylan Antos, Kelly James Vanee

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

A dock device connects participating devices such as a tablet device and an audio activated device, allowing them to operate as a single device. These participating devices may be associated with different accounts, each account being associated with particular “speechlets” or data processing functions. A natural language understanding (NLU) system uses NLU models to process text obtained from an automatic speech recognition (ASR) system to determine a set of possible intents. A second set of possible intents may then be generated that is limited to those possible intents that correspond to the speechlets associated with the docked device. The intents within the second set of possible intents are ranked, and the highest ranked intent may be deemed to be the intent of the user. Command data corresponding to the highest ranked intent may be generated and used to perform the action associated with that intent.

Claims21

Dependent claims

A system comprising:

at least one memory storing computer-executable instructions;
and
at least one processor in communication with the at least one memory, the at least one processor executing the computer-executable instructions to:
determine a first device is associated with a user account;
determine a first speechlet associated with the user account, wherein the first speechlet comprises a first set of data processing functions available to the first device;
determine a second device is associated with the user account;
determine a second speechlet associated with the user account, wherein the second speechlet comprises a second set of data processing functions available to the second device;
generate a speechlet set comprising the first set of data processing functions and the second set of data processing functions;
determine a third device that is associated with the first device;
determine an output capability that is indicative of a type of output that the third device is able to present;
receive first data from the first device;
process the first data using a first natural language understanding (NLU) model to determine a set of possible intents that are representative of intended actions as expressed in the first data that are available in the speechlet set;
based at least in part on the output capability, determine a first intent of the set of possible intents as a first ranked intent associated with performing an action;
generate command data corresponding to the first ranked intent;
and
send the command data to another device.

13 more elements

A system comprising:

at least one memory storing computer-executable instructions;
and
at least one processor in communication with the at least one memory, the at least one processor executing the computer-executable instructions to:
determine a first device is associated with a user account;
determine a second device is associated with the user account;
determine speechlet data indicative of one or more speechlets available to the first device and the second device to process one or more intents;
determine a third device that is associated with the first device;
determine an output capability that is indicative of a type of output that the third device is able to present;
receive first data from one or more of the first device or the second device;
process the first data using a first natural language understanding (NLU) model to determine a set of possible intents that are representative of intended actions as expressed in the first data that are available to the one or more speechlets indicated by the speechlet data;
rank the set of possible intents;
based at least in part on the output capability, select, from the set of possible intents, a first ranked intent associated with performing an intended action;
and
generate command data corresponding to the first ranked intent.

11 more elements

A method comprising:

determining a first device is associated with a user account;
determining a second device is associated with the user account;
determining a third device that is associated with the first device;
determining speechlet data that is indicative of one or more data processing functions available to the first device and the second device to process one or more intents;
determining an output capability that is indicative of a type of output that the third device is able to present;
receiving first data from one or more of the first device or the second device;
processing the first data using one or more natural language understanding (NLU) models to determine a set of possible intents that are representative of intended actions as expressed in the first data that are available to the one or more data processing functions indicated by the speechlet data;
based at least in part on the output capability and from the set of possible intents, determining a first ranked intent associated with performing an intended action;
and
generating command data corresponding to the first ranked intent.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/716,353Filed 26 Sep 2017
Granted: 10 Dec 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification12 codes

G10L 15/22
Speech recognition: procedures used during a speech recognition process, e.g. man-machine dialogue
G10L 15/02
Speech recognition: feature extraction for speech recognition; Selection of recognition unit
G10L 15/1815
Using natural language modelling: semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
G10L 15/30
Constructional details of speech recognition systems: distributed recognition, e.g. in client-server systems, for mobile phones or network…

Family

No related US applications are recorded in the indexed filing history.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key