# US 2019/0147044 A1 · Underspecification of intents in a natural language processing system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0147044 A1
- Document ID: US20190147044A1
- Status: Published application
- Kind: A1
- Application number: 16/248,433
- Filed: 2019-01-15
- Published: 2019-05-16
- Earliest claimed: 2016-12-19
- Assignees: Interactions LLC
- Applicants: Interactions LLC
- Inventors: Srinivas Bangalore; John Chen
- Web page: https://patentgrep.com/patent/US20190147044A1

## CPC

- G06F 40/30: Semantic analysis
- G06F 16/3329: Natural language query formulation
- G06F 40/137: Hierarchical processing, e.g. outlines
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/183: using context dependencies, e.g. language models
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A natural language processing system has a hierarchy of user intents related to a domain of interest, the hierarchy having specific intents corresponding to leaf nodes of the hierarchy, and more general intents corresponding to ancestor nodes of the leaf nodes. The system also has a trained understanding model that can classify natural language utterances according to user intent. When the understanding model cannot determine with sufficient confidence that a natural language utterance corresponds to one of the specific intents, the natural language processing system traverses the hierarchy of intents to find a more general user intent that is related to the most applicable specific intent of the utterance and for which there is sufficient confidence. The general intent can then be used to prompt the user with questions applicable to the general intent to obtain the missing information needed for a specific intent.

## Claims

### Claim 1

A non-transitory computer-readable storage medium comprising instructions executable by a processor for identifying an intent associated with a natural language utterance, the instructions comprising:
instructions for receiving a natural language utterance of a user;
instructions for accessing a domain hierarchy of intents comprising leaf nodes and ancestor nodes;
instructions for accessing a plurality of intent understanding models, each of the plurality of intent understanding models associated with one of the nodes of the domain hierarchy of intents, wherein an intent understanding model is trained, through supervised machine learning based on a training set of utterances that are labeled with user intents, to determine a confidence score for the associated node;
instructions for producing a plurality of node confidence scores by applying the plurality of intent understanding models of the nodes to the natural language utterance;
instructions for determining that none of the node confidence scores corresponding to the leaf nodes exceeds a given minimum confidence threshold;
instructions for identifying a most applicable ancestor node based on confidence scores of ancestor nodes; and
instructions for identifying, as a general intent of the natural language utterance, an intent corresponding to the identified most applicable ancestor node.

### Claim 2 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, the instructions further comprising:
instructions for identifying, based on the identified general intent, a prompt for additional information; and
instructions for providing the prompt to the user over the computer network.

### Claim 3 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein any descendant node in the hierarchy of intents corresponds to a more specific user intent than the intents corresponding to any ancestor nodes from which the descendant node is descended.

### Claim 4 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein applying the intent understanding models comprises associating the natural language utterance with a verb from a domain hierarchy of actions and with a noun from a domain hierarchy of intents related to items.

### Claim 5 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein applying the intent understanding models comprises associating the natural language utterance with two or more intents, each intent selected from its own intent hierarchy.

### Claim 6 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the ancestor node confidence score is determined by calculating a function of the confidence scores of other nodes in the same intent hierarchy.

### Claim 7 (depends on 6)

The non-transitory computer-readable storage medium of claim 6, wherein the ancestor node confidence score is determined by calculating a geometric mean of the leaf node confidence scores of leaf nodes descended from the ancestor node.

### Claim 8 (depends on 6)

The non-transitory computer-readable storage medium of claim 6, wherein the ancestor node confidence score is determined by determining a maximum confidence score of leaf nodes descended from the ancestor node.

### Claim 9 (depends on 6)

The non-transitory computer-readable storage medium of claim 6, the instructions further comprising:
instructions for applying, to the utterance, a plurality of hierarchy understanding models for a corresponding plurality of domain hierarchies to produce a corresponding plurality of scores for the utterance;
instructions for selecting a subset of the domain hierarchies based on the scores; and
instructions for identifying the most applicable ancestor node based on the selected subset.

### Claim 10

A computer-implemented method for identifying an intent associated with a natural language utterance comprising:
receiving a natural language utterance of a user over a computer network;
accessing a domain hierarchy of intents comprising leaf nodes and ancestor nodes;
accessing a plurality of intent understanding models, each of the plurality of intent understanding models associated with one of the nodes of the domain hierarchy of intents, wherein an intent understanding model is trained, through supervised machine learning based on a training set of utterances that are labeled with user intents, to determine a confidence score for the associated node;
producing a plurality of node confidence scores by applying the plurality of intent understanding models of the nodes to the natural language utterance;
determining that none of the node confidence scores corresponding to the leaf nodes exceeds a given minimum confidence threshold;
identifying a most applicable ancestor node based on confidence scores of ancestor nodes; and
identifying, as a general intent of the natural language utterance, an intent corresponding to the identified most applicable ancestor node.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, further comprising:
identifying, based on the identified general intent, a prompt for additional information; and
providing the prompt to the user over the computer network.

### Claim 12 (depends on 10)

The computer-implemented method of claim 10, wherein any descendant node in the hierarchy of intents corresponds to a more specific user intent than the intents corresponding to any ancestor nodes from which the descendant node is descended.

### Claim 13 (depends on 10)

The computer-implemented method of claim 10, wherein applying the intent understanding models comprises associating the natural language utterance with a verb from a domain hierarchy of actions and with a noun from a domain hierarchy of intents related to items.

### Claim 14 (depends on 10)

The computer-implemented method of claim 10, wherein applying the intent understanding models comprises associating the natural language utterance with two or more intents, each intent selected from its own intent hierarchy.

### Claim 15 (depends on 10)

The computer-implemented method of claim 10, wherein the ancestor node confidence score is determined by calculating a function of the confidence scores of other nodes in the same intent hierarchy.

### Claim 16 (depends on 15)

The computer-implemented method of claim 15, wherein the ancestor node confidence score is determined by calculating a geometric mean of the leaf node confidence scores of leaf nodes descended from the ancestor node.

### Claim 17 (depends on 15)

The computer-implemented method of claim 15, wherein the ancestor node confidence score is determined by determining a maximum confidence score of leaf nodes descended from the ancestor node.

### Claim 18 (depends on 15)

The computer-implemented method of claim 15, further comprising:
applying, to the utterance, a plurality of hierarchy understanding models for a corresponding plurality of domain hierarchies to produce a corresponding plurality of scores for the utterance;
selecting a subset of the domain hierarchies based on the scores; and
identifying the most applicable ancestor node based on the selected subset.
