Active learning for spoken language understanding

A non-transitory computer-readable storage medium storing instructions for controlling a computing device to generate a spoken language understanding module, the instructions comprising, from a small amount of training data St and a larger amount of unlabeled data Su:

• (1) training a plurality of classifiers independently using a training data set St, the training data St generated by sampling a plurality of utterances;
• (2) classifying utterances in a set Su using the plurality of classifiers and computing a call type confidence score for all utterances;
• (3) sorting candidate utterances with respect to a score of the maximum scoring call type according to one of the classifiers if the classifiers disagree;
• (4) selecting and labeling the lowest scored k utterances from Su to define a labeled set Si and redefining St and Su as follows:
• (5) St=St∪Si;
• and
• (6) Su=Su−Si, wherein the labeled utterances are used to generate the spoken language understanding module.

A method of generating a spoken dialog understanding module, the method causing a processor of a computing device to perform steps comprising, from a small amount of training data St and a larger amount of unlabeled data Su:

• classifying via the processor of the computing device utterances in an unlabelled data set Su using a plurality of classifiers;
• computing via the processor of the computing device a call type confidence score for all utterances;
• selecting utterances for labeling from the unlabeled data Su based on whether the classification from the plurality of classifiers disagree;
• redefining St=St∪a labeled set Si;
• redefining Su=Su−Si
• labeling the selected utterances;
• and
• generating a spoken language understanding module using the labeled utterances.