US 11,081,104 B1Granted

Contextual natural language processing

AssigneeAmazon Technologies, Inc.

InventorsChengwei Su, Sankaranarayanan Ananthakrishnan, Spyridon Matsoukas, Shirin Saleem, Rahul Gupta, Kavya Ravikumar, John Will Crimmins, Kelly James Vanee, John Pelak, Melanie Chie Bomke Gens

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

A natural language understanding system that can determine an overall score for a natural language hypothesis using hypothesis-specific component scores from different aspects of NLU processing as well as context data describing the context surrounding the utterance corresponding to the natural language hypotheses. The individual component scores may be input into a feature vector at a location corresponding to a type of a device captured by the utterance. Other locations in the feature vector corresponding to other device types may be populated with zero values. The feature vector may also be populated with other values represent other context data. The feature vector may then be multiplied by a weight vector comprising trained weights corresponding to the feature vector positions to determine a new overall score for each hypothesis, where the overall score incorporates the impact of the context data. Natural language hypotheses can be ranked using their respective new overall scores.

Claims32

Dependent claims

A computer-implemented method comprising:

receiving, from a first device, input audio data corresponding to an utterance;
performing speech recognition processing on the input audio data to generate text data representing the utterance;
determining a natural language hypothesis representing a potential interpretation of the text data, wherein determining the natural language hypothesis comprises:
processing the text data using an intent classifier to generate an intent score representing a likelihood that an intent corresponds to the text data,
processing the text data using a named entity recognition (NER) model to identify a portion of the text data potentially relevant to the intent,
processing the text data using the NER model to generate a NER score indicating a likelihood the text data is relevant to the intent, and
processing the text data using a domain classification model to generate a domain score representing a likelihood that the text data corresponds to a domain;
determining a device type of the first device;
determining a feature vector comprising the intent score, the NER score and the domain score, wherein:
the intent score is located at a first position in the feature vector, the first position corresponding to at least the device type,
the NER score is located at a second position in the feature vector, the second position corresponding to at least the device type, and
the domain score is located at a third position in the feature vector, the third position corresponding to at least the device type;
sending the feature vector to a log linear model component comprising a weight vector, the weight vector comprising a plurality of weights, wherein the plurality of weights comprises:
a first weight corresponding to the first position,
a second weight corresponding to the second position, and
a third weight corresponding to the third position;
multiplying the intent score by the first weight to determine a weighted intent score;
multiplying the NER score by the second weight to determine a weighted NER score;
multiplying the domain score by the third weight to determine a weighted domain score;
determining a new score for the natural language hypothesis using at least the weighted intent score, the weighted NER score and the weighted domain score;
and
causing a command corresponding to the intent to be executed.

19 more elements

A system comprising:

at least one processor;
and
at least one memory including instructions that, when executed by the at least one processor, cause the system to:
receive, from a device, a device identifier of the device and input data corresponding to a user input;
process text data representing the input data to determine a first plurality of scores for a first natural language hypothesis corresponding to the input data;
determine, based on at least the device identifier, that the input data corresponds to a first context profile;
populate a first feature vector with the first plurality of scores, the first plurality of scores being at a first plurality of positions in the first feature vector, the first plurality of positions corresponding to the first context profile, wherein a second plurality of positions in the first feature vector corresponds to a second context profile;
multiply the first feature vector by a weight vector including a first plurality of weights associated with the first context profile and a second plurality of weights associated with a second context profile to determine first result data;
determine a first new score for the first natural language hypothesis using the first result data;
and
select the first natural language hypothesis for further operations based at least in part on the first new score.

8 more elements

A computer-implemented method comprising:

receiving, from a device, a device identifier of the device and input data corresponding to a user input;
processing text data representing the input data to determine a first plurality of scores for a first natural language hypothesis corresponding to the input data;
determining, based on at least the device identifier, that the input data corresponds to a first context profile;
populating a first feature vector with the first plurality of scores, the first plurality of scores being at a first plurality of positions in the first feature vector, the first plurality of positions corresponding to the first context profile, wherein a second plurality of positions in the first feature vector corresponds to a second context profile;
multiplying the first feature vector by a weight vector including a first plurality of weights associated with the first context profile and a second plurality of weights associated with a second context profile to determine first result data;
determining a first new score for the first natural language hypothesis using the first result data;
and
selecting the first natural language hypothesis for further operations based at least in part on the first new score.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/838,917Filed 12 Dec 2017 · continuation in part
Priority: 27 Jun 2017
Granted: 3 Aug 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification21 codes

G10L 15/063
Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice: training
G06F 3/167
Sound input; Sound output: audio in a user interface, e.g. using voice commands for navigating, audio feedback
G06F 16/313
Indexing; Data structures therefor; Storage structures: selection or weighting of terms for indexing
G06F 40/216
Parsing: using statistical methods