Domain-specific chatbot utterance collection
AssigneeFigure Eight Technologies, Inc.
InventorsPhoebe Liu, Caiqun Xiao, Tong Liu
Life of this patent
Estimating the term…
Abstract
A set of utterances collected from a plurality of contributors is received. Semantically irrelevant utterances are removed from the set of utterances to obtain a processed set of utterances, including by applying a machine learning model to the set of utterances. An annotation user interface is provided to a plurality of human annotators to perform annotation on the processed set of utterances to obtain an annotated set of utterances. A curation user interface is provided to one or more domain experts to perform curation of the annotated set of utterances to obtain a curated set of utterances. The curated set of utterances is outputted as a training set for an automated dialogue agent.
Claims20
A method, comprising:
- receiving a set of utterances collected from a plurality of contributors;
- removing one or more semantically irrelevant utterances from the set of utterances to obtain a processed set of utterances, including by applying a machine learning model to determine how semantically similar an utterance in the set of utterances is to a set of one or more baseline utterances;
- providing an annotation user interface to a plurality of human annotators to perform annotation on the processed set of utterances to obtain an annotated set of utterances;
- providing a curation user interface to one or more domain experts to perform curation of the annotated set of utterances to obtain a curated set of annotated utterances;
- and
- outputting the curated set of annotated utterances as a training set for an automated dialogue agent.
A system, comprising:
- a processor configured to:
- receive a set of utterances collected from a plurality of contributors;
- remove one or more semantically irrelevant utterances from the set of utterances to obtain a processed set of utterances, including by applying a machine learning model to determine how semantically similar an utterance in the set of utterances is to a set of one or more baseline utterances;
- provide an annotation user interface to a plurality of human annotators to perform annotation on the processed set of utterances to obtain an annotated set of utterances;
- provide a curation user interface to one or more domain experts to perform curation of the annotated set of utterances to obtain a curated set of annotated utterances;
- and
- output the curated set of annotated utterances as a training set for an automated dialogue agent;
- and
- a memory coupled to the processor and configured to provide the processor with instructions.
A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
- receiving a set of utterances collected from a plurality of contributors;
- removing one or more semantically irrelevant utterances from the set of utterances to obtain a processed set of utterances, including by applying a machine learning model to determine how semantically similar an utterance in the set of utterances is to a set of one or more baseline utterances;
- providing an annotation user interface to a plurality of human annotators to perform annotation on the processed set of utterances to obtain an annotated set of annotated utterances;
- providing a curation user interface to one or more domain experts to perform curation of the annotated set of utterances to obtain a curated set of utterances;
- and
- outputting the curated set of annotated utterances as a training set for an automated dialogue agent.
Dependencies are inferred from the claim text.