# US 2023/0267328 A1 · Matching based intent understanding with transfer learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267328 A1
- Document ID: US20230267328A1
- Status: Published application
- Kind: A1
- Application number: 18/310,242
- Filed: 2023-05-01
- Published: 2023-08-24
- Earliest claimed: 2019-03-12
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Jianshu JI; Yeyun GONG; Nan DUAN; Yi-Cheng PAN; Guihong CAO
- Web page: https://patentgrep.com/patent/US20230267328A1

## CPC

- G06F 40/30: Semantic analysis
- G06N 3/08: Learning methods
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 3/096: Transfer learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 5/04: Inference or reasoning models
- G06N 20/00: Machine learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Described herein is a mechanism to identify user intent in requests submitted to a system such as a digital assistant or question-answer systems. Embodiments utilize a match methodology instead of a classification methodology. Features derived from a subgraph retrieved from a knowledge base based on the request are concatenated with pretrained word embeddings for both the request and a candidate predicate. The concatenated inputs for both the request and predicate are encoded using two independent LSTM networks and then a matching score is calculated using a match LSTM network. The result is identified based on the matching scores for a plurality of candidate predicates. The pretrained word embeddings allow for knowledge transfer since pretrained word embeddings in one intent domain can apply to another intent domain without retraining.

## Claims

### Claim 1

A method of a digital assistant system detecting and actioning user intent in natural language requests, comprising:
receiving a request from a user;
identifying a candidate predicate based on the request;
retrieving a subgraph from a knowledge base based on the request;
concatenating features derived from the subgraph with pretrained word embeddings to yield a set of request inputs and a set of predicate inputs;
calculating a matching score for the request and candidate predicate using a trained machine learning model based on the set of request inputs and the set of predicate inputs;
selecting a matching predicate comprising user intent based on the matching score;
performing an action to effectuate the user intent; and
outputting a response to the user.

### Claim 2 (depends on 1)

The method of claim 1 wherein the trained machine learning model comprises a first trained bi-directional LSTM neural network and a second trained bi-directional LSTM network.

### Claim 3 (depends on 1)

The method of claim 1 wherein the trained machine learning model comprises a trained bi-directional matching LSTM neural network.

### Claim 4 (depends on 3)

The method of claim 3 wherein the trained machine learning model further comprises a first trained bi-directional LSTM network utilizing the set of request inputs and a second trained bi-directional LSTM network utilizing the set of predicate inputs.

### Claim 5 (depends on 1)

The method of claim 1 wherein the set of request inputs comprises word embedding based on the request concatenated with a subset of the features derived from the subgraph.

### Claim 6 (depends on 1)

The method of claim 1 wherein the set of predicate inputs comprises word embedding based on the candidate predicate concatenated with a subset of the features derived from the subgraph.

### Claim 7 (depends on 1)

The method of claim 1 wherein the trained machine learning model comprises a self-attention layer.

### Claim 8 (depends on 1)

The method of claim 1 wherein the trained machine learning model comprises a sigmoid layer.

### Claim 9 (depends on 1)

The method of claim 1 wherein the pretrained word embeddings for a first intent domain also apply to a second intent domain without retraining.

### Claim 10 (depends on 1)

The method of claim 1 wherein retrieving a subgraph from a knowledge base based on the request comprises:
detecting an entity in the request;
retrieving the subgraph from the knowledge base based on the entity;
deriving the features from the subgraph using a convolutional neural network.

### Claim 11

A digital assistant system comprising a processor and computer executable instructions, that when executed by the processor, cause the system to perform operations comprising:
receive a request from a user;
identify a candidate predicate based on the request;
retrieve a subgraph from a knowledge base based on the request;
deriving a set of features from the subgraph using a convolutional neural network;
concatenate features from the set of features with pretrained word embeddings to yield a set of request inputs and a set of predicate inputs;
calculate a matching score for the request and candidate predicate using a trained machine learning model based on the set of request inputs and the set of predicate inputs;
select a matching predicate comprising user intent based on the matching score;
perform an action to effectuate the user intent; and
output a response to the user.

### Claim 12 (depends on 11)

The system of claim 11 wherein the trained machine learning model comprises a first trained bi-directional LSTM neural network and a second trained bi-directional LSTM network.

### Claim 13 (depends on 11)

The system of claim 11 wherein the trained machine learning model comprises a trained bi-directional matching LSTM neural network.

### Claim 14 (depends on 13)

The system of claim 13 wherein the trained machine learning model further comprises a first trained bi-directional LSTM network utilizing the set of request inputs and a second trained bi-directional LSTM network utilizing the set of predicate inputs.

### Claim 15 (depends on 11)

The system of claim 11 wherein the set of request inputs comprises word embedding based on the request concatenated with a subset of the features derived from the subgraph.

### Claim 16 (depends on 11)

The system of claim 11 wherein the set of predicate inputs comprises word embedding based on the candidate predicate concatenated with a subset of the features derived from the subgraph.

### Claim 17 (depends on 11)

The system of claim 11 wherein the trained machine learning model comprises a self-attention layer.

### Claim 18 (depends on 11)

The system of claim 11 wherein the trained machine learning model comprises a sigmoid layer.

### Claim 19 (depends on 11)

The system of claim 11 wherein the pretrained word embeddings for a first intent domain also apply to a second intent domain without retraining.

### Claim 20

A computer storage medium comprising executable instructions that, when executed by a processor of a machine, cause the machine to perform operations of a digital assistant system, the operations comprising:
receive a request from a user;
identify a candidate predicate based on the request;
identifying an entity in the request;
retrieve a subgraph from a knowledge base based on the entity;
deriving a set of features from the subgraph using a convolutional neural network;
concatenate features from the set of features with pretrained word embeddings to yield a set of request inputs and a set of predicate inputs;
calculate a matching score for the request and candidate predicate using a trained machine learning model based on the set of request inputs and the set of predicate inputs;
select a matching predicate comprising user intent based on the matching score;
performing an action to effectuate the user intent; and
outputting a response to the user.
