Learning dialogue state tracking with limited labeled data
Assigneesalesforce.com, inc.
InventorsChien-Sheng Wu, Chu Hong Hoi, Caiming Xiong
Life of this patent
Estimating the term…
Abstract
Embodiments described in this disclosure illustrate the use of self-/semi supervised approaches for label-efficient DST in task-oriented dialogue systems. Conversational behavior is modeled by next response generation and turn utterance generation tasks. Prediction consistency is strengthened by augmenting data with stochastic word dropout and label guessing. Experimental results show that by exploiting self-supervision the joint goal accuracy can be boosted with limited labeled data.
Claims20
A method for boosting prediction consistency in a dialogue state tracking (DST) module, the method comprising:
- receiving, from a database, a first dataset of dialogue data;
- generating, via the DST module, a first predicted attention distribution and a first predicted slot gate distribution from the first dataset of dialogue data;
- generating multiple datasets of masked dialogue data by masking one or more words in the first dataset of dialogue data over multiple iterations;
- generating, via the DST module, a set of predicted attention distributions and predicted slot gate distributions from the masked dialogue data;
- computing an average predicted attention distribution and an average predicted slot gate distribution from the first dataset of dialogue data and the multiple datasets of masked dialogue data;
- computing a consistency loss metric based on a first error metric between the average predicted attention distribution and the first predicted attention distribution obtained from unmasked dialogue data, and a second error metric between the average predicted slot gate distribution and the first predicted slot gate distribution obtained from unmasked dialogue data;
- and
- incorporating the consistency loss metric into a loss metric for updating the DST module.
A system for boosting prediction consistency in a dialogue state tracking (DST) module, the system comprising:
- a memory storing instructions for boosting prediction consistency in the DST module;
- and
- a processor configured to read and execute the instructions from the memory to:
- receive, from a database, a first dataset of dialogue data;
- generate, via the DST module, a first predicted attention distribution and a first predicted slot gate distribution from the first dataset of dialogue data;
- generate multiple datasets of masked dialogue data by masking one or more words in the first dataset of dialogue data over multiple iterations;
- generate, via the DST module, a set of predicted attention distributions and predicted slot gate distributions from the masked dialogue data;
- compute an average predicted attention distribution and an average predicted slot gate distribution from the first dataset of dialogue data and the multiple datasets of masked dialogue data;
- compute a consistency loss metric based on a first error metric between the average predicted attention distribution and the first predicted attention distribution obtained from unmasked dialogue data, and a second error metric between the average predicted slot gate distribution and the first predicted slot gate distribution obtained from unmasked dialogue data;
- and
- incorporate the consistency loss metric into a loss metric for updating the DST module.
A non-transitory processor-readable storage medium storing processor-executable instructions for boosting prediction consistency in a dialogue state tracking (DST) module, the instructions being executed by a processor to:
- receive, from a database, a first dataset of dialogue data;
- generate, via the DST module, a first predicted attention distribution and a first predicted slot gate distribution from the first dataset of dialogue data;
- generate multiple datasets of masked dialogue data by masking one or more words in the first dataset of dialogue data over multiple iterations;
- generate, via the DST module, a set of predicted attention distributions and predicted slot gate distributions from the masked dialogue data;
- compute an average predicted attention distribution and an average predicted slot gate distribution from the first dataset of dialogue data and the multiple datasets of masked dialogue data;
- compute a consistency loss metric based on a first error metric between the average predicted attention distribution and the first predicted attention distribution obtained from unmasked dialogue data, and a second error metric between the average predicted slot gate distribution and the first predicted slot gate distribution obtained from unmasked dialogue data;
- and
- incorporate the consistency loss metric into a loss metric for updating the DST module.
Dependencies are inferred from the claim text.