Training call routing applications by reusing semantically-labeled data collected for prior applications

A computer system for semantic classification, the computer system comprising:

• a processor;
• and
• a memory coupled to the processor, the memory storing instructions that when executed by the processor cause the system to perform the operations of:
• receiving a first utterance for use in training a first automatic semantic classifier, the first automatic semantic classifier having a set of semantic classes for labeling transcribed utterances, the first utterance being a transcribed utterance classified in a first semantic class within the set of semantic classes;
• semantically labeling the first utterance using a second automatic semantic classifier, the second automatic semantic classifier being a previously-trained semantic classifier;
• semantically labeling existing utterances using the second automatic semantic classifier, the existing utterances accessed from a database of existing utterances, the database of existing utterances containing transcribed utterances used to train previously-trained automatic semantic classifiers;
• identifying existing utterances that are semantically similar to the first utterance based on semantic labeling from the second automatic classifier;
• and
• training the first automatic semantic classifier using the semantically similar existing utterances.

A computer-implemented method of semantic classification, the computer-implemented method comprising:

• receiving, at a computerized training manager, a set of utterances for training a given computerized semantic classifier, the set of utterances being transcribed utterances;
• semantically labeling utterances from the set of utterances by multiple previously-trained computerized semantic classifiers;
• identifying, by the computerized training manager, semantically labeled utterances that are semantically similar;
• grouping, by the computerized training manager, utterances from the semantically labeled utterances into sets of utterances based on a predetermined measure of semantic similarity between pairs of semantically labeled utterances;
• presenting, by the computerized training manager, a given utterance from the set of utterances along with available semantic labels for manual selection of an available semantic label to classify a given set of semantically grouped utterances;
• and
• training, by the computerized training manager, the given computerized semantic classifier using the classified given set of semantically grouped utterances.

A computer system for semantic classification, the computer system comprising:

• a processor;
• and
• a memory coupled to the processor, the memory storing instructions that when executed by the processor cause the system to perform the operations of:
• receiving a set of utterances for training a given automatic semantic classifier, the set of utterances being transcribed utterances;
• semantically labeling utterances from the set of utterances using multiple previously-trained automatic semantic classifiers;
• identifying semantically labeled utterances that are semantically similar;
• grouping utterances from the semantically labeled utterances into sets of utterances based on a measure of semantic similarity between pairs of semantically labeled utterances;
• presenting a given utterance from the set of utterances along with available semantic labels for manual selection of an available semantic label to classify a given set of semantically grouped utterances;
• and
• training the given automatic semantic classifier using the classified given set of semantically grouped utterances.

A computer-implemented method of semantic classification, the computer-implemented method comprising:

• labeling sentences from a first group of sentences by multiple different computerized text classifiers to produce a label and produce a classification confidence corresponding to the produced label, wherein each of the multiple different computerized text classifiers produces a respective label and classification pairing for each respective sentence from the first group of sentences, the first group of sentences including sentences collected from sample sentences used for training existing text classifiers used with speech recognition applications, the label representing a sentence meaning;
• labeling sentences from a second group of sentences by the multiple different computerized text classifiers to produce a label and produce a classification confidence corresponding to the produced label, wherein each of the multiple different computerized text classifiers produces a respective label and classification pairing for each respective sentence from the second group of sentences, the second group of sentences including sentences collected from sample training sentences for training a new computerized text classifier used with a speech recognition application;
• computing, by a computerized training manager, a semantic similarity for each labeled sentence from the second group of sentences as compared to each labeled sentence from the first group of sentences;
• identifying, by the computerized training manager, labeled sentences from the first group of sentences that meet a predetermined measure of similarity to each given labeled sentence from the second group of sentences;
• grouping, by the computerized training manager, identified labeled sentences from the first group of sentences with corresponding sentences from the second group of sentences based on the predetermined measure of similarity;
• and
• training, by the computerized training manager, the new text classifier used with the speech recognition application using identified labeled sentences from the first group of sentences and sentences from a second group of sentences, the speech recognition application being a call routing application.

A computer-implemented method of semantic classification, the computer-implemented method comprising:

• receiving, at a computerized training manager, a first set of semantic labels;
• receiving, at a computerized training manager, a first set of sentences that includes at least one sample sentence for each semantic label from the first set of semantic labels;
• creating, by the computerized training manager, a database of sentences that includes sentences used as training sentences to train text classifiers used with speech recognition applications;
• identifying, by the computerized training manager, multiple different computerized text classifiers trained for classifying text as part of speech recognition applications, each of multiple different computerized text classifiers, when executed with a given input sentence, produces a semantic classification label and corresponding classification confidence of the produced semantic classification label for the given input sentence;
• semantically labeling sentences from the database of sentences by the multiple different computerized text classifiers, wherein each of the multiple different computerized text classifiers produces a respective semantic label and classification pairing for each respective sentence from the database of sentences;
• semantically labeling sentences from the first set of sentences by the multiple different computerized text classifiers, wherein each of the multiple different computerized text classifiers produces a respective semantic label and classification pairing for each respective sentence from the first set of sentences;
• computing, by the computerized training manager, a semantic similarity for each semantically labeled sentence from the first group of sentences as compared to each semantically labeled sentence from the database of sentences;
• for each semantically labeled sentence from the first set of sentences, identifying, by the computerized training manager, semantically labeled sentences from the database of sentences that meet a predetermined measure of similarity;
• adding, by the computerized training manager, the identified semantically labeled sentences from the database of sentences as sample sentences to the first set of sentences according to the computed semantic similarity and the first set of semantic labels;
• and
• training, by the computerized training manager, a statistical language model according to the first set of semantic labels using the identified semantically labeled sentences from the database of sentences and the first set of sentences.

A computer-implemented method of semantic classification, the computer-implemented method comprising:

• receiving, at a computerised training manner, a first set of sample training sentences for training a text classifier, the sample training sentences requiring semantic labeling;
• identifying, by the computerized training manager, multiple different computerized text classifiers trained for classifying text as part of speech recognition applications, each of multiple different computerized text classifiers, when executed with a given input sentence, produces a semantic classification label and corresponding classification confidence of the produced classification label for the given input sentence;
• labeling sentences from the first set of sample training sentences by the multiple different computerized text classifiers, wherein each of the multiple different computerized text classifiers produces a respective semantic label and classification pairing for each respective sentence from the first set of sample training sentences;
• computing, by the computerized training manager, a semantic similarity for each labeled sentence from the first set of sample training sentences as compared to other labeled sentences from the first set of sample training sentences;
• hierarchically grouping, by the computerized training manager, labeled sentences from the first set of sample training sentences using a complete linkage algorithm based on a predetermined measure of similarity, each group within the hierarchy of labeled sentences corresponding to a semantic label;
• presenting, by the computerized training manager, a given sentence from the first set of sample training sentences with at least one available semantic label, the at least one available semantic label corresponding to a group within the hierarchy or labeled sentences;
• and
• training, by the computerized training manager, the text classifier using the group within the hierarchy of labeled sentences.