Evaluating text classification anomalies predicted by a text classification model

A computer system comprising one or more processors, one or more computer-readable memories, one or more computer-readable storage devices, and program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:

• program instructions to run a testing phrase on a previously trained text classifier, wherein the testing phrase comprises one or more words;
• program instructions to identify a separate predicted classification label based on an intent score calculated for the testing phrase by the text classifier;
• program instructions to decompose a plurality of extracted features summed in the score into a plurality of word-level scores for each word in the testing phrase;
• program instructions to assign a separate heatmap value to each of the plurality of word-level scores, each respective separate heatmap value reflecting a weight of each word-level score of the plurality of word-level scores;
• and
• program instructions to output the separate predicted classification label and each separate heatmap value reflecting the weight of each respective word-level score of the plurality of word-level scores, for defining a heatmap identifying the contribution of each word in the testing phrase to the separate predicted classification label.

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the computer readable storage medium is not a transitory signal per se, the program instructions executable by a computer to cause the computer to:

• run, by a computer, at least one testing phrase on a previously trained text classifier, wherein the testing phrase comprises one or more words;
• identify, by a computer, a separate predicted classification label based on an intent score calculated for the testing phrase by the text classifier;
• decompose, by a computer, a plurality of extracted features summed in the score into a plurality of word-level scores for each word in the testing phrase;
• assign, by the computer, a separate heatmap value to each of the plurality of word-level scores, each respective separate heatmap value reflecting a weight of each word-level score of the plurality of word-level scores;
• and
• output, by the computer, the separate predicted classification label and each separate heatmap value reflecting the weight of each respective word-level score of the plurality of word-level scores, for defining a heatmap identifying the contribution of each word in the testing phrase to the separate predicted classification label.