Systems and methods for model comparison and evaluation

A system for comparing a plurality of models, comprising:

• one or more processors;
• one or more non-transitory computer readable media storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
• generating raw scores for the plurality of models based on multiple measures of demographic bias and performance, wherein each of the raw scores is associated with a corresponding model of the plurality of models and a corresponding measure of the multiple measures of demographic bias and performance;
• storing the raw scores for each of the plurality of models in corresponding locations of a raw score matrix, wherein each of the locations of the raw score matrix is associated with a corresponding measure of the multiple measures of demographic bias and performance and a corresponding model of the plurality of models;
• determining rank scores for the plurality of models with respect to the multiple measures of demographic bias and performance, the determining based on comparing the raw scores of the plurality of models in each of the multiple measures of demographic bias and performance;
• storing the rank scores for each of the plurality of models in corresponding locations of a rank matrix, wherein each of the locations of the rank matrix is associated with a corresponding measure of the multiple measures of demographic bias and performance and a corresponding model of the plurality of models;
• determining comparative scores for the plurality of models based on a pairwise comparison of the rank scores of the plurality of models, wherein the pairwise comparison of the rank scores includes a statistical comparison of two random variables to select at least one model from the plurality of models;
• storing the comparative scores in corresponding locations of a comparative matrix, wherein each of the locations of the comparative matrix is associated with a corresponding model of the plurality of models and represents a win, a loss, or a draw against another model of the plurality of models, the comparative scores including a stored win representing a first model compared to a second model and a stored loss representing the first model compared to a third model, the comparative scores representing stored wins having a first shared value and the comparative scores representing stored losses having a second shared value;
• determining a rank for each of the plurality of models based on tallying the comparative scores of the comparative matrix and one or more of a model run time or complexity of interpretability of a model;
• causing to display a model associated with a first rank indicative of a least biased model to a user via a user interface based on the rank determined for each of the plurality of models;
• and
• causing an automated deployment of the model associated with the first rank and prevention of deployment of at least one model associated with a subsequent rank indicative of a biased model.

One or more non-transitory computer readable media for comparing a plurality of models, the one or more non-transitory computer readable media storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• generating raw scores for the plurality of models based on multiple measures of demographic bias and performance, wherein each of the raw scores is associated with a corresponding model of the plurality of models and a corresponding measure of the multiple measures of demographic bias and performance;
• storing the raw scores for each of the plurality of models in corresponding locations of a raw score matrix, wherein each of the locations of the raw score matrix is associated with a corresponding measure of the multiple measures of demographic bias and performance and a corresponding model of the plurality of models;
• determining rank scores for the plurality of models with respect to the multiple measures of demographic bias and performance, the determining based on comparing the raw scores of the plurality of models in each of the multiple measures of demographic bias and performance;
• storing the rank scores for each of the plurality of models in corresponding locations of a rank matrix, wherein each of the locations of the rank matrix is associated with a corresponding measure of the multiple measures of demographic bias and performance and a corresponding model of the plurality of models;
• determining comparative scores for the plurality of models based on a pairwise comparison of the rank scores of the plurality of models, wherein the pairwise comparison of the rank scores includes a statistical comparison of two random variables to select at least one model from the plurality of models;
• storing the comparative scores in corresponding locations of a comparative matrix, wherein each of the locations of the comparative matrix is associated with a corresponding model of the plurality of models and represents a win, a loss, or a draw against another model of the plurality of models, the comparative scores including a stored win representing a first model compared to a second model and a stored loss representing the first model compared to a third model, the comparative scores representing stored wins having a first shared value and the comparative scores representing stored losses having a second shared value;
• determining a rank for each of the plurality of models based on tallying the comparative scores of the comparative matrix and one or more of a model run time or complexity of interpretability of a model;
• causing to display a model associated with a first rank indicative of a least biased model to a user via a user interface based on the rank determined for each of the plurality of models;
• and
• causing an automated deployment of the model associated with the first rank and prevention of deployment of at least one model associated with a subsequent rank indicative of a biased model.