Analytic system for graphical interpretability of and improvement of machine learning models

A computing device comprising:

• a processor;
• and
• a non-transitory computer-readable medium operably coupled to the processor, the computer-readable medium having computer-readable instructions stored thereon that, when executed by the processor, cause the computing device to
• read a dataset that includes a plurality of observation vectors, wherein each observation vector of the plurality of observation vectors includes an explanatory variable value for an explanatory variable and a response variable value for a response variable;
• train a model using the plurality of observation vectors;
• predict a classification value for the response variable for each observation vector of the plurality of observation vectors using a respective explanatory variable value input to the trained model;
• divide the plurality of observation vectors into a plurality of overlapping data slices;
• separately cluster each data slice of the plurality of overlapping data slices using the predicted classification value to define a set of clusters for the plurality of observation vectors;
• compute a number of observations in each cluster of the set of clusters;
• compute an accuracy measure for each cluster of the set of clusters based on a comparison between the predicted classification value and a respective response variable value of each observation vector of the plurality of observation vectors;
• compute a number of overlapping observations between each pair of clusters of the set of clusters;
• and
• present a cluster connectivity graph on a display, wherein the cluster connectivity graph includes a node for each cluster of the set of clusters, wherein a size of each node is determined from the computed number of observations, wherein a fill-pattern of each node is determined from the computed accuracy measure, wherein a connector line between each pair of nodes is determined from the computed number of overlapping observations.

A method provides a visualization that summarizes a performance of a machine learning model to support model improvement and understanding, the method comprising:

• reading, by a computing device, a dataset that includes a plurality of observation vectors, wherein each observation vector of the plurality of observation vectors includes an explanatory variable value for an explanatory variable and a response variable value for a response variable;
• training, by the computing device, a model using the plurality of observation vectors;
• predicting, by the computing device, a classification value for the response variable for each observation vector of the plurality of observation vectors using a respective explanatory variable value input to the trained model;
• dividing, by the computing device, the plurality of observation vectors into a plurality of overlapping data slices;
• separately clustering, by the computing device, each data slice of the plurality of overlapping data slices using the predicted classification value to define a set of clusters for the plurality of observation vectors;
• computing, by the computing device, a number of observations in each cluster of the set of clusters;
• computing, by the computing device, an accuracy measure for each cluster of the set of clusters based on a comparison between the predicted classification value and a respective response variable value of each observation vector of the plurality of observation vectors;
• computing, by the computing device, a number of overlapping observations between each pair of clusters of the set of clusters;
• and
• presenting, by the computing device, a cluster connectivity graph on a display, wherein the cluster connectivity graph includes a node for each cluster of the set of clusters, wherein a size of each node is determined from the computed number of observations, wherein a fill-pattern of each node is determined from the computed accuracy measure, wherein a connector line between each pair of nodes is determined from the computed number of overlapping observations.