System for visually diagnosing machine learning models

A computer-implemented method, comprising:

• performing, by a model diagnosis system implemented on one or more processors and associated memory:
• obtaining prediction results of a ML model that classifies a given media sample to one or more of a plurality of classes, wherein the ML model was trained using training samples from a training set and the prediction results were generated using test samples from another set;
• generating a model performance interface with a zoomable confusion matrix that groups the test samples into cells according to their respective truth classes and predicted classes;
• responsive to user input via the model performance interface to select a cell of the confusion matrix, causing the model performance interface to zoom in on test samples in the cell selected, wherein the test samples represent classification errors by the ML model;
• responsive to user input to select one or more of the test samples in the cell, generating a model diagnosis interface configured to display the one or more test samples with added visual information for a visual diagnosis of the classification errors;
• receiving user input via the model diagnosis interface indicating feedback regarding the one or more test samples selected or one or more training samples in the training set;
• determining, based at least in part on the feedback, one or more modifications to the training set or the ML model to improve prediction performance;
• generating a diagnosis report interface configured to display the one or more modifications to the training set or the ML model and to perform the one or more modifications via one or more user control elements;
• and
• responsive to user input receive via the diagnosis report interface, performing at least one of the one or more modifications on the training set or the ML model.

One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors implement a model diagnosis system for machine learning (ML) models and cause the model diagnosis system to:

• obtain prediction results of a ML model that classifies a given media sample to one or more of a plurality of classes, wherein the ML model was trained using training samples from a training set and the prediction results were generated using test samples from another set;
• generate a model performance interface with a zoomable confusion matrix that groups the test samples into cells according to their respective truth classes and predicted classes;
• responsive to user input via the model performance interface to select a cell of the confusion matrix, cause the model performance interface to zoom in on test samples in the cell selected, wherein the test samples represent classification errors by the ML image model;
• responsive to user input to select one or more of the test samples in the cell, generate a model diagnosis interface configure to display the one or more test samples with added visual information for a visual diagnosis of the classification errors;
• receive user input via the model diagnosis interface indicating feedback regarding the one or more test samples selected or one or more training samples in the training set;
• determine, based at least in part on the feedback, one or more modifications to the training set or the ML model to improve prediction performance;
• generate a diagnosis report interface configured to display the one or more modifications to the training set or the ML model and to perform the one or more modifications via one or more user control elements;
• and
• responsive to user input receive via the diagnosis report interface, perform at least one of the one or more modifications on the training set or the ML model.