System and method for improving performance of semantic classifiers in spoken dialog systems

A spoken dialog system, comprising:

• a computer including a processor, and memory, including:
• a signal input for receiving an audio input;
• a speech recognition engine;
• a data store comprising an initial set of semantic classifiers;
• a data store for a plurality of utterances received via the audio input;
• a data store for storing transcribed utterances;
• a data store for storing annotated utterances;
• a semantic classifier component including a semantic classifier program for, when executed by the processor, mapping the utterances to a set of semantic classes;
• a quality assurance component including a program for, when executed by the processor, applying quality assurance criteria to the annotated utterances;
• a classifier update generation component including a program for, when executed by the processor, generating an update candidate for the initial set of semantic classifiers using data from the annotated utterances if a quality assurance criterion is met;
• and a comparison component including a program for, when executed by the processor, comparing a performance of the update candidate against the initial set of semantic classifiers by testing the update candidate and the initial set of semantic classifiers against a baseline criterion, whereby the initial set of semantic classifiers is updated with the update candidate if the update candidate outperforms the initial set of classifiers,
• wherein the quality assurance criteria are selected from at least two or more of the group consisting of:
• completeness of the annotated utterances;
• consistency of the annotated utterances;
• congruence of the annotated utterances;
• correlation of the annotated utterances;
• confusion of the annotated utterances;
• coverage of the annotated utterances;
• and
• corpus size of the annotated utterances.