# US 2018/0174578 A1 · Underspecification of intents in a natural language processing system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0174578 A1
- Document ID: US20180174578A1
- Status: Published application
- Kind: A1
- Application number: 15/384,275
- Filed: 2016-12-19
- Published: 2018-06-21
- Earliest claimed: 2016-12-19
- Assignees: Interactions LLC
- Applicants: Interactions LLC
- Inventors: SRINIVAS BANGALORE; John Chen
- Web page: https://patentgrep.com/patent/US20180174578A1

## CPC

- G06F 40/30: Semantic analysis
- G06F 16/3329: Natural language query formulation
- G06F 40/137: Hierarchical processing, e.g. outlines
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/183: using context dependencies, e.g. language models
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A natural language processing system has a hierarchy of user intents related to a domain of interest, the hierarchy having specific intents corresponding to leaf nodes of the hierarchy, and more general intents corresponding to ancestor nodes of the leaf nodes. The system also has a trained understanding model that can classify natural language utterances according to user intent. When the understanding model cannot determine with sufficient confidence that a natural language utterance corresponds to one of the specific intents, the natural language processing system traverses the hierarchy of intents to find a more general user intent that is related to the most applicable specific intent of the utterance and for which there is sufficient confidence. The general intent can then be used to prompt the user with questions applicable to the general intent to obtain the missing information needed for a specific intent.

## Claims

### Claim 1

A non-transitory computer-readable storage medium comprising instructions executable by a processor, the instructions comprising:
instructions for receiving a natural language utterance specified by a user;
instructions for accessing a domain hierarchy of intents comprising leaf nodes and ancestor nodes, wherein each of a plurality of the leaf nodes has a corresponding intent understanding model trained through supervised machine learning based on respective training sets of utterances that are labeled with user intents that the sets of utterances are known to represent;
instructions for applying a plurality of intent understanding models of the leaf nodes to the natural language utterance, thereby producing a corresponding plurality of leaf node confidence scores;
instructions for determining that none of the leaf node confidence scores exceeds a given minimum confidence threshold;
instructions for, responsive to the determining, determining ancestor node confidence scores for the ancestor nodes;
instructions for identifying a most applicable ancestor node based on the ancestor node confidence scores; and
instructions for identifying, as a general intent of the natural language utterance, an intent corresponding to the identified most applicable ancestor node.

### Claim 2 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, the instructions further comprising:
instructions for identifying, based on the identified general intent, a prompt for additional information; and
instructions for providing the prompt to the user over the computer network.

### Claim 3 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein any descendant node in the hierarchy of intents corresponds to a more specific user intent than the intents corresponding to any ancestor nodes from which the descendant node is descended.

### Claim 4 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein applying the intent understanding models comprises associating the natural language utterance with a verb from a domain hierarchy of actions and with a noun from a domain hierarchy of intents related to items.

### Claim 5 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein applying the intent understanding models comprises associating the natural language utterance with two or more intents, each intent selected from its own intent hierarchy.

### Claim 6 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the ancestor node confidence score is determined by calculating a function of the confidence scores of other nodes in the same intent hierarchy.

### Claim 7 (depends on 6)

The non-transitory computer-readable storage medium of claim 6, wherein the ancestor node confidence score is determined by calculating a geometric mean of the leaf node confidence scores of leaf nodes descended from the ancestor node.

### Claim 8 (depends on 6)

The non-transitory computer-readable storage medium of claim 6, wherein the ancestor node confidence score is determined by determining a maximum confidence score of leaf nodes descended from the ancestor node.

### Claim 9 (depends on 6)

The non-transitory computer-readable storage medium of claim 6, wherein the ancestor node confidence score is determined by:
identifying an intent understanding model corresponding to the ancestor node; and
applying the identified intent understanding model to the natural language utterance.

### Claim 10

A computer-implemented method comprising:
receiving a natural language utterance specified by a user over a computer network;
accessing a domain hierarchy of intents comprising leaf nodes and ancestor nodes, wherein each of a plurality of the leaf nodes has a corresponding intent understanding model trained through supervised machine learning based on respective training sets of utterances that are labeled with user intents the sets of utterances are known to represent;
applying the intent understanding models corresponding to each of the plurality of leaf nodes to the natural language utterance, thereby producing a corresponding plurality of leaf node confidence scores;
determining that none of the leaf node confidence scores exceeds a given minimum confidence threshold;
responsive to the determining, determining ancestor node confidence scores for the ancestor nodes;
identifying a most applicable ancestor node based on the ancestor node confidence scores; and
identifying, as a general intent of the natural language utterance, an intent corresponding to the identified most applicable ancestor node.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, further comprising:
identifying, based on the identified general intent, a prompt for additional information; and
providing the prompt to the user over the computer network.

### Claim 12 (depends on 10)

The computer-implemented method of claim 10, wherein the leaf nodes in the hierarchy of intents correspond to more specific user intents than the intents corresponding to the ancestor nodes from which the leaf nodes are descended.

### Claim 13 (depends on 10)

The computer-implemented method of claim 10, wherein applying the intent understanding models comprises associating the natural language utterance with a verb from a domain hierarchy of actions and with a noun from a domain hierarchy of intents related to items.

### Claim 14 (depends on 10)

The computer-implemented method of claim 10, wherein applying the intent understanding models comprises associating the natural language utterance with two or more intents, each intent selected from its own intent hierarchy.

### Claim 15 (depends on 10)

The computer-implemented method of claim 10, wherein the ancestor node confidence score is determined by calculating a function of the confidence scores of other nodes in the same intent hierarchy.

### Claim 16 (depends on 15)

The computer-implemented method of claim 15, wherein the ancestor node confidence score is determined by calculating a geometric mean of the leaf node confidence scores of leaf nodes descended from the ancestor node.

### Claim 17 (depends on 15)

The computer-implemented method of claim 15, wherein the ancestor node confidence score is determined by:
identifying an intent understanding model corresponding to the ancestor node; and
applying the identified intent understanding model to the natural language utterance.

### Claim 18 (depends on 15)

The computer-implemented method of claim 15, further comprising:
applying, to the utterance, a plurality of hierarchy understanding models for a corresponding plurality of domain hierarchies to produce a corresponding plurality of scores for the utterance;
selecting a subset of the domain hierarchies based on the scores; and
identifying the most applicable ancestor node based on the selected subset.
