Model performance simulator

A method for determining a predicted model performance metric, the method comprising:

• receiving a model associated with a development population, the model configured to:
• receive an input population comprising a plurality of credit card applicants, each credit card applicant being associated with a credit score;
• and
• generate an output corresponding to a percentage of the input population associated with a past due credit card balance;
• receiving an input population filter;
• applying the filter to at least a portion of the development population;
• determining a plurality of performance metrics based on comparing:
• (1) the output generated by applying the filter to at least a portion of the development population;
• and
• (2) a target number of number of accounts associated with the past due credit card balance;
• and
• adjusting the filter when at least one of the simulated performance metrics is associated with a shifting of the output that is greater than a threshold value deviation from the target number.

A method of calculating a predicted performance metric (“PPM”) of a model, the method comprising:

• receiving a model development population;
• receiving a target model performance metric;
• receiving a set of values corresponding to a filter;
• applying the filter to the model development population;
• based on the applying, calculating:
• a first percentage of the development population associated with a first characteristic;
• and
• a second percentage of the development population associated with a second characteristic;
• calculating the PPM based on a first difference between:
• the first percentage;
• and
• the second percentage;
• comparing the PPM to the target model performance metric;
• determining if a second difference between the PPM and the target model performance metric is less than a threshold difference;
• and
• when the second difference is less than the threshold difference, applying the filter to incoming model population.

A model performance simulator that is configured to predict an accuracy of a model output, the simulator comprising:

• a non-transitory computer readable medium having computer readable program code embodied therein;
• and
• a processor configured to execute the computer readable program code;
• the computer readable program code comprising:
• computer readable code for causing the simulator to receive a plurality of values corresponding to a filter;
• computer readable code for causing the simulator to determine, prior to deployment of the model, a simulated effect of integrating the filter into the model;
• computer readable code for causing the simulator to calculate a model performance metric based on the simulated effect of the integrating;
• computer readable code for causing the simulator compare the model performance metric to a target model performance metric;
• and
• computer readable code for causing the simulator, when a difference between the model performance metric and the target performance metric exceeds a threshold, to associate the filter with a risk of an inaccurate model output.