Systems and methods for automatic bias monitoring of cohort models and un-deployment of biased models

A computer-implemented method for automatically assessing a machine learning model for selection of a cohort, the method comprising:

• training a machine learning model to select individuals for a cohort, the machine learning model including a neural network and being trained using a training set of a plurality of labeled medical records, wherein each labeled medical record of the plurality of labeled medical records:
• includes a plurality of patient characteristics associated with a corresponding individual, and
• is labeled to indicate a desired outcome for whether the corresponding individual is included in a cohort;
• inputting information associated with a first plurality of individuals into the machine learning model;
• receiving, from the machine learning model, an indication of the cohort comprising a selected subset of the first plurality of individuals, wherein a quantity of the selected subset of the first plurality of individuals is determined at least partially based on a sensitivity threshold of the machine learning model;
• identifying, from a testing set of the plurality of labeled medical records a second plurality of individuals, wherein the plurality of labeled medical records indicate the desired outcome for the second plurality of individuals is inclusion in the cohort;
• determining, among the selected subset, a distribution of the first plurality of individuals relative to at least one characteristic of the plurality of patient characteristics, based on the information associated with the first plurality of individuals;
• determining, among the second plurality of individuals, a distribution of the second plurality of individuals relative to the at least one characteristic based on the testing set of the plurality of labeled medical records;
• comparing the distribution of the first plurality of individuals and the distribution of the second plurality of individuals to determine a value indicating a difference between the distribution of the first plurality of individuals and the distribution of the second plurality of individuals;
• detecting a bias of the machine learning model based on a comparison of the value to a difference threshold;
• in response to the detected bias, generate a report indicating the detected bias, the report including a graph correlating the sensitivity of the machine learning model to a specificity of the machine learning model;
• and
• adjusting the sensitivity threshold of the machine learning model based on the detected bias.

One or more non-transitory computer readable storage media storing program instructions which, when executed by at least one processing device, cause the at least one processing device to perform a method for automatically assessing a machine learning model for selection of a cohort, the method comprising:

• training a machine learning model to select individuals for a cohort, the machine learning model including a neural network and being trained using a training set of a plurality of labeled medical records, wherein each labeled medical record of the plurality of labeled medical records:
• includes a plurality of patient characteristics associated with a corresponding individual, and
• is labeled to indicate a desired outcome for whether the corresponding individual is included in a cohort;
• inputting information associated with a first plurality of individuals into the machine learning model;
• receiving, from the machine learning model, an indication of the cohort comprising a selected subset of the first plurality of individuals, wherein a quantity of the selected subset of the first plurality of individuals is determined at least partially based on a sensitivity threshold of the machine learning model;
• identifying, from a testing set of the plurality of labeled medical records a second plurality of individuals, wherein the plurality of labeled medical records indicate the desired outcome for the second plurality of individuals is inclusion in the cohort;
• determining, among the selected subset, a distribution of the first plurality of individuals relative to at least one characteristic of the plurality of patient characteristics, based on the information associated with the first plurality of individuals;
• determining, among the second plurality of individuals, a distribution of the second plurality of individuals relative to the at least one characteristic based on the testing set of the plurality of labeled medical records;
• comparing the distribution of the first plurality of individuals and the distribution of the second plurality of individuals to determine a value indicating a difference between the distribution of the first plurality of individuals and the distribution of the second plurality of individuals;
• detecting a bias of the machine learning model based on a comparison of the value to a difference threshold;
• in response to the detected bias, generate a report indicating the detected bias, the report including a graph correlating the sensitivity of the machine learning model to a specificity of the machine learning model;
• and
• adjusting the sensitivity threshold of the machine learning model based on the detected bias.