Utterance annotation user interface

A method, comprising:

• graphically displaying an unannotated transcript representing an ordered sequence of one or more dialogue events involving a client and a computerized assistant, at least one of the dialogue events taking the form of an example client utterance;
• graphically displaying a hierarchical menu including a plurality of candidate utterance annotations;
• receiving one or more computer inputs selecting, for each of one or more response parameters in the example client utterance, one or more utterance annotations from the hierarchical menu, the one or more selected utterance annotations collectively defining a machine-readable interpretation of the example client utterance;
• and
• outputting, to a data store, an annotated utterance based on the example client utterance and each of the one or more selected utterance annotations, the annotated utterance having a predetermined format usable to train the computerized assistant.

A computing device, comprising:

• a display engine configured to present a graphical user interface on a display device, the graphical user interface comprising:
• a transcript portion configured to display an unannotated transcript representing an ordered sequence of one or more dialogue events involving a client and a computerized assistant, at least one of the dialogue events taking the form of an example client utterance;
• and
• an annotation portion configured to display a hierarchical menu including a plurality of candidate utterance annotations, each of the plurality of candidate utterance annotations being associated with one or more category-specific domains, such that selection of a category-specific domain exposes one or more candidate utterance annotations associated with the selected category-specific domain;
• and
• an utterance annotation machine configured to:
• receive one or more computer inputs selecting, for each of one or more response parameters in the example client utterance, one or more utterance annotations from the hierarchical menu, the one or more selected utterance annotations collectively defining a machine-readable interpretation of the example client utterance;
• generate and display a computerized assistant response to the example client utterance based on the one or more selected utterance annotations;
• receive one or more computer inputs modifying the one or more selected utterance annotations to address inconsistencies between the example client utterance and the computerized assistant response;
• and
• output, to a data store, an annotated utterance based on the example client utterance and each of the one or more selected utterance annotations, the annotated utterance having a predetermined format usable to train the computerized assistant.