Rapid Adjustment Evaluation for Slow-Scoring Machine Learning Models

A method for system for evaluating training data changes on a machine learning model, the method comprising:

• training a first instance of the machine learning model with a first set of training data;
• modifying the first set of training data to remove one or more data elements, add one or more data elements, or both to produce a second set of training data;
• training a second instance of the machine learning model with the second set of training data;
• comparing the first instance of the machine learning model to the second instance of the machine learning model to determine features that differ between the first instance of the machine learning model and the second instance of the machine learning model;
• identifying a subset of historical data associated with the features that differ between the first instance of the machine learning model and the second instance of the machine learning model;
• and
• scoring the subset of the historical data to produce a report identifying differences in the output of the first instance of the machine learning model and the second instance of the machine learning model.

A memory device storing instructions that, when executed on a processor of a data processing system, cause the data processing system to evaluating training data changes on a machine learning model, by:

• training a first instance of the machine learning model with a first set of training data;
• modifying the first set of training data to remove one or more data elements, add one or more data elements, or both to produce a second set of training data;
• training a second instance of the machine learning model with the second set of training data;
• comparing the first instance of the machine learning model to the second instance of the machine learning model to determine features that differ between the first instance of the machine learning model and the second instance of the machine learning model;
• identifying a subset of historical data associated with the features that differ between the first instance of the machine learning model and the second instance of the machine learning model;
• and
• scoring the subset of the historical data to produce a report identifying differences in the output of the first instance of the machine learning model and the second instance of the machine learning model.