# US 2021/0174026 A1 · Learning dialogue state tracking with limited labeled data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174026 A1
- Document ID: US20210174026A1
- Status: Published application
- Kind: A1
- Application number: 16/870,568
- Filed: 2020-05-08
- Published: 2021-06-10
- Earliest claimed: 2019-12-09
- Assignees: salesforce.com, inc.
- Applicants: Salesforce.com, Inc.
- Inventors: Chien-Sheng Wu; Chu Hong Hoi; Caiming Xiong
- Web page: https://patentgrep.com/patent/US20210174026A1

## CPC

- G06F 40/35: Discourse or dialogue representation
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 5/04: Inference or reasoning models
- G10L 15/16: using artificial neural networks
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Embodiments described in this disclosure illustrate the use of self-/semi supervised approaches for label-efficient DST in task-oriented dialogue systems. Conversational behavior is modeled by next response generation and turn utterance generation tasks. Prediction consistency is strengthened by augmenting data with stochastic word dropout and label guessing. Experimental results show that by exploiting self-supervision the joint goal accuracy can be boosted with limited labeled data.

## Claims

### Claim 1

A method for self-supervised training of a dialogue state tracking (DST) module, the method comprising:
receiving, from a database, a first input of unlabeled dialogue data including a time sequence of dialogue pairs, each dialogue pair including a respective unlabeled user utterance and a respective unlabeled system response;
generating, via the DST module, a predicted system response subsequent to a first dialogue pair based on a first set of dialogue pairs that occur by and include the first dialogue pair in the time sequence of dialogue pairs and a predicted dialogue state from the first dialogue pair;
generating, via the DST module, a predicted user utterance corresponding to the first dialogue pair based on a second set of dialogue pairs that occur prior to the first dialogue pair in the time sequence of dialogue pairs and the predicted dialogue state from the first dialogue pair;
computing an unsupervised loss metric based on the predicted system response and the generated user utterance for the first dialogue pair; and
updating the DST module based on the unsupervised loss metric.

### Claim 2 (depends on 1)

The method of claim 1, wherein the predicted system response is generated, at a first dialogue turn associated with the first dialogue pair, as a conditional probability distribution of the predicted system response for a subsequent dialogue turn conditioned on the first set of dialogue pairs and the predicted dialogue state inferred from the first dialogue pair at the first dialogue turn.

### Claim 3 (depends on 1)

The method of claim 1, wherein the predicted user utterance is generated, at a first dialogue turn associated with the first dialogue pair, as a conditional probability distribution of the predicted current user utterance for the first dialogue turn conditioned on the second set of dialogue pairs and the predicted dialogue state inferred from the first dialogue pair at the first dialogue turn.

### Claim 4 (depends on 1)

The method of claim 1, wherein the predicted system response or the predicted user utterance is generated by a gated recurrent unit initialized by information from a context encoder that encodes the first input of unlabeled dialogue data.

### Claim 5 (depends on 4)

The method of claim 4, wherein the gated recurrent unit receives a state corresponding to a last hidden layer of the context encoder as an initial state.

### Claim 6 (depends on 1)

The method of claim 1, wherein the predicted system response or the predicted user utterance is generated by a gated recurrent unit initialized by information from a state generator that generates state information for each dialogue pair in the time sequence of the dialogue pairs.

### Claim 7 (depends on 6)

The method of claim 6, wherein the gated recurrent unit receives hidden states of the state generator queried by each domain-slot pair and a respective context vector corresponding to the respective domain-slot pair, and generates each output word conditioned on the hidden states of the context encoder.

### Claim 8 (depends on 1)

The method of claim 1, wherein the unsupervised loss metric is computed as a sum of a first cross entropy between the predicted system response and an actual system response from a second dialogue pair subsequent to the first dialogue pair, and a second cross entropy between the generated user utterance and an actual user utterance from the first dialogue pair.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
receiving a second input of labeled dialogue data;
computing a supervised loss metric for the DST module from the second input of labeled dialogue data;
computing the unsupervised loss metric for the DST module from the second input of labeled dialogue data but without dialogue labels;
computing a weighted sum of the supervised loss metric and the unsupervised metric; and
updating the DST module based on the weighted sum of the supervised loss metric and the unsupervised loss metric.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
alternately training the DST module by:
using the weighted sum of the supervised loss metric and the unsupervised loss metric derived from labeled dialogue data; and
using the unsupervised loss metric derived from unlabeled dialogue data.

### Claim 11

A system for self-supervised training of a dialogue state tracking (DST) module, the method comprising:
a memory storing instructions for self-supervised training of the DST module; and
a processor configured to read and execute the instructions from the memory to:
receive, from a database, a first input of unlabeled dialogue data including a time sequence of dialogue pairs, each dialogue pair including a respective unlabeled user utterance and a respective unlabeled system response;
generate, via the DST module, a predicted system response subsequent to a first dialogue pair based on a first set of dialogue pairs that occur by and include the first dialogue pair in the time sequence of dialogue pairs and a predicted dialogue state from the first dialogue pair;
generate, via the DST module, a predicted user utterance corresponding to the first dialogue pair based on a second set of dialogue pairs that occur prior to the first dialogue pair in the time sequence of dialogue pairs and the predicted dialogue state from the first dialogue pair;
compute an unsupervised loss metric based on the predicted system response and the generated user utterance for the first dialogue pair; and
update the DST module based on the unsupervised loss metric.

### Claim 12 (depends on 11)

The system of claim 11, wherein the predicted system response is generated, at a first dialogue turn associated with the first dialogue pair, as a conditional probability distribution of the predicted system response for a subsequent dialogue turn conditioned on the first set of dialogue pairs and the predicted dialogue state inferred from the first dialogue pair at the first dialogue turn.

### Claim 13 (depends on 11)

The system of claim 11, wherein the predicted user utterance is generated, at a first dialogue turn associated with the first dialogue pair, as a conditional probability distribution of the predicted current user utterance for the first dialogue turn conditioned on the second set of dialogue pairs and the predicted dialogue state inferred from the first dialogue pair at the first dialogue turn.

### Claim 14 (depends on 11)

The system of claim 11, wherein the predicted system response or the predicted user utterance is generated by a gated recurrent unit initialized by information from a context encoder that encodes the first input of unlabeled dialogue data.

### Claim 15 (depends on 14)

The system of claim 14, wherein the gated recurrent unit receives a state corresponding to a last hidden layer of the context encoder as an initial state.

### Claim 16 (depends on 11)

The system of claim 11, wherein the predicted system response or the predicted user utterance is generated by a gated recurrent unit initialized by information from a state generator that generates state information for each dialogue pair in the time sequence of the dialogue pairs.

### Claim 17 (depends on 16)

The system of claim 16, wherein the gated recurrent unit receives hidden states of the state generator queried by each domain-slot pair and a respective context vector corresponding to the respective domain-slot pair, and generates each output word conditioned on the hidden states of the context encoder.

### Claim 18 (depends on 11)

The system of claim 11, wherein the unsupervised loss metric is computed as a sum of a first cross entropy between the predicted system response and an actual system response from a second dialogue pair subsequent to the first dialogue pair, and a second cross entropy between the generated user utterance and an actual user utterance from the first dialogue pair.

### Claim 19 (depends on 11)

The system of claim 11, wherein the processor is further configured to read and execute the instructions from the memory to:
receive a second input of labeled dialogue data;
compute a supervised loss metric for the DST module from the second input of labeled dialogue data;
compute the unsupervised loss metric for the DST module from the second input of labeled dialogue data but without dialogue labels;
compute a weighted sum of the supervised loss metric and the unsupervised metric;
update the DST module based on the weighted sum of the supervised loss metric and the unsupervised loss metric; and
alternately train the DST module by:
using the weighted sum of the supervised loss metric and the unsupervised loss metric derived from labeled dialogue data; and
using the unsupervised loss metric derived from unlabeled dialogue data.

### Claim 20

A non-transitory processor-readable storage medium storing processor-executable instructions for self-supervised training of a dialogue state tracking (DST) module, the instructions being executed by a processor to:
receive, from a database, a first input of unlabeled dialogue data including a time sequence of dialogue pairs, each dialogue pair including a respective unlabeled user utterance and a respective unlabeled system response;
generate, via the DST module, a predicted system response subsequent to a first dialogue pair based on a first set of dialogue pairs that occur by and include the first dialogue pair in the time sequence of dialogue pairs and a predicted dialogue state from the first dialogue pair;
generate, via the DST module, a predicted user utterance corresponding to the first dialogue pair based on a second set of dialogue pairs that occur prior to the first dialogue pair in the time sequence of dialogue pairs and the predicted dialogue state from the first dialogue pair;
compute an unsupervised loss metric based on the predicted system response and the generated user utterance for the first dialogue pair; and
update the DST module based on the unsupervised loss metric.
