# US 12,223,275 B2 · Method of training model, device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,275 B2
- Document ID: US12223275B2
- Status: Granted
- Kind: B2
- Application number: 17/559,623
- Filed: 2021-12-22
- Published: 2022-04-14
- Granted: 2025-02-11
- Earliest claimed: 2020-12-25
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science Technology Co., Ltd.
- Inventors: Weijie Ren; Jianfei Wang; Cheng Peng
- Web page: https://patentgrep.com/patent/US12223275B2

## CPC

- G06F 40/295: Named entity recognition
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 40/30: Semantic analysis
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06F 16/3329: Natural language query formulation
- G06F 16/35: Clustering; Classification
- G10L 15/063: {Training}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method of training a model, an electronic device, and a storage medium are provided, which relate to a field of artificial intelligence, and in particular to a field of reinforcement learning, NLP, etc. The method includes: acquiring a dialogue information; obtaining a predicted information based on the dialogue information by using a first intention recognition model, a first recurrent neural network and a first named entity recognition model; obtaining a machine behavior information based on the predicted information by using a first behavior decision model; acquiring a feedback information for the machine behavior; storing at least one of the predicted information, the machine behavior information, or the feedback information as training data in a database; and performing a model optimization training online based on the training data by using a reinforcement learning algorithm, in response to an amount of the training data reaching a preset data amount.

## Claims

### Claim 1

A method of training a model, comprising:
acquiring a dialogue information input by a user in a human-machine interaction process;
obtaining a corresponding predicted information based on the dialogue information by using a first intention recognition model, a first recurrent neural network, and a first named entity recognition model, wherein the predicted information includes: a user intention information, a dialogue state information and an entity information;
obtaining a corresponding machine behavior information based on the predicted information by using a first behavior decision model, in order for a machine to perform a corresponding machine behavior in the human-machine interaction process based on the machine behavior information;
acquiring a feedback information input by the user for the machine behavior;
storing at least one of the predicted information, the machine behavior information, or the feedback information as training data in a database; and
performing a model optimization training online based on the training data in the database by using a reinforcement learning algorithm, in response to an amount the training data in the database reaching a preset data amount, wherein
the performing a model optimization training online based on the training data in the database by using a reinforcement learning algorithm, including, performing the model optimization training online on at least one of the first intention recognition model, the first recurrent neural network, the first named entity recognition model, or the first behavior decision model, based on the training data in the database by using the reinforcement learning algorithm, and
the method further includes at least one of:
performing an optimization training on a fully connected layer of the first intention recognition model, in performing the model optimization training on the first intention recognition model; and
performing an optimization training on a fully connected layer of the first behavior decision model, in performing the model optimization training on the first behavior decision model.

### Claim 2 (depends on 1)

The method of claim 1, wherein the performing a model optimization training online based on the training data in the database by using a reinforcement learning algorithm, comprising:
selecting training data randomly from the database; and
performing the model optimization training online based on the training data selected randomly, by using the reinforcement learning algorithm.

### Claim 3 (depends on 1)

The method of claim 1, wherein the obtaining a corresponding predicted information based on the dialogue information by using a first intention recognition model, a first recurrent neural network and a first named entity recognition model, comprising at least one of:
obtaining a probability distribution of a user intention based on the dialogue information by using the first intention recognition model;
obtaining a probability distribution of a dialogue state based on the dialogue information by using the first recurrent neural network; and
obtaining a corresponding entity vector based on the dialogue information by using the first named entity recognition model.

### Claim 4 (depends on 2)

The method of claim 2, wherein the obtaining a corresponding predicted information based on the dialogue information by using a first intention recognition model, a first recurrent neural network and a first named entity recognition model, comprising at least one of:
obtaining a probability distribution of a user intention based on the dialogue information by using the first intention recognition model;
obtaining a probability distribution of a dialogue state based on the dialogue information by using the first recurrent neural network; and
obtaining a corresponding entity vector based on the dialogue information by using the first named entity recognition model.

### Claim 5 (depends on 1)

The method of claim 1, wherein the obtaining a corresponding machine behavior information based on the predicted information, by using a first behavior decision model, comprising:
obtaining a probability distribution of the corresponding machine behavior based on the predicted information by using the first behavior decision model.

### Claim 6 (depends on 1)

An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to implement the method of claim 1.

### Claim 7 (depends on 6)

The electronic device of claim 6, wherein the at least one processor is further configured to:
select training data randomly from the database; and
perform the model optimization training online based on the training data selected randomly, by using the reinforcement learning algorithm.

### Claim 8 (depends on 6)

The electronic device of claim 6, wherein the at least one processor is further configured to perform at least one of:
obtaining a probability distribution of a user intention based on the dialogue information by using the first intention recognition model;
obtaining a probability distribution of a dialogue state based on the dialogue information by using the first recurrent neural network; and
obtaining a corresponding entity vector based on the dialogue information by using the first named entity recognition model.

### Claim 9 (depends on 6)

The electronic device of claim 6, wherein the at least one processor is further configured to:
obtain a probability distribution of the corresponding machine behavior based on the predicted information by using the first behavior decision model.

### Claim 10 (depends on 1)

A non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are configured to cause a computer to implement the method of claim 1.

### Claim 11 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein the computer instructions are further configured to cause a computer to:
select training data randomly from the database; and
perform the model optimization training online based on the training data selected randomly, by using the reinforcement learning algorithm.

### Claim 12 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein the computer instructions are configured to cause a computer to perform at least one of:
obtaining a probability distribution of a user intention based on the dialogue information by using the first intention recognition model;
obtaining a probability distribution of a dialogue state based on the dialogue information by using the first recurrent neural network; and
obtaining a corresponding entity vector based on the dialogue information by using the first named entity recognition model.

### Claim 13 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein the computer instructions are configured to cause a computer to:
obtain a probability distribution of the corresponding machine behavior based on the predicted information by using the first behavior decision model.