Computer-based visualization of machine-learning models and behavior

A system comprising:

• a processing device;
• and
• a memory device on which instructions executable by the processing device are stored for causing the processing device to:
• teach a machine-learning model using a teaching dataset that has pieces of input data correlated to output categories, wherein each piece of input data is correlated to a particular output category and is formed from multiple elements;
• determine a plurality of weights for a plurality of elements in the teaching dataset, wherein each weight in the plurality of weights corresponds to a respective element in the plurality of elements and is determined based on a relationship between (i) a first value indicating a number of pieces of input data in the teaching dataset that both include the respective element and belong to a specific output category, and (ii) a second value indicating a proportion of a total number of pieces of input data in the teaching data set that have the respective element;
• generate an analysis dataset that includes a subset of the plurality of elements that have corresponding weights that are above a predefined threshold;
• provide a test input to the machine-learning model to determine a baseline confidence-score of the machine-learning model, the test input having one or more elements in the analysis dataset;
• for each overlapping element in both the analysis dataset and the test input:
• generate a modified version of the test input that excludes the overlapping element;
• provide the modified version of the test input to the machine-learning model to determine an effect of the overlapping element on the baseline confidence-score;
• and
• add the overlapping element to (i) a first dataset in response to the effect of the overlapping element on the baseline confidence-score being positive, or (ii) a second dataset in response to the effect of the overlapping element on the baseline confidence-score being negative;
• and
• generate a graphical user interface that visually depicts:
• the test input;
• the first dataset as having a positive effect on the baseline confidence-score;
• and
• the second dataset as having a negative effect on the baseline confidence-score.

A non-transitory computer-readable medium comprising program code that is executable by a processing device for causing the processing device to:

• teach a machine-learning model using a teaching dataset that has pieces of input data correlated to output categories, wherein each piece of input data is correlated to a particular output category and is formed from multiple elements;
• determine a plurality of weights for a plurality of elements in the teaching dataset, wherein each weight in the plurality of weights corresponds to a respective element in the plurality of elements and is determined based on a relationship between (i) a first value indicating a number of pieces of input data in the teaching dataset that both include the respective element and belong to a specific output category, and (ii) a second value indicating a proportion of a total number of pieces of input data in the teaching data set that have the respective element;
• generate an analysis dataset that includes a subset of the plurality of elements that have corresponding weights that are above a predefined threshold;
• provide a test input to the machine-learning model to determine a baseline confidence-score of the machine-learning model, the test input having one or more elements in the analysis dataset;
• for each overlapping element in both the analysis dataset and the test input:
• generate a modified version of the test input that excludes the overlapping element;
• provide the modified version of the test input to the machine-learning model to determine an effect of the overlapping element on the baseline confidence-score;
• and
• add the overlapping element to (i) a first dataset in response to the effect of the overlapping element on the baseline confidence-score being positive, or (ii) a second dataset in response to the effect of the overlapping element on the baseline confidence-score being negative;
• and
• generate a graphical user interface that visually depicts:
• the test input;
• the first dataset as having a positive effect on the baseline confidence-score;
• and
• the second dataset as having a negative effect on the baseline confidence-score.