A computer-implemented method for determining surrogate ground truth to enable fairness evaluation after completion of a campaign of interest, the surrogate ground truth indicating individuals who should have been contacted by the campaign of interest, the computer-implemented method comprising:
- receiving, by a computing device, data for the campaign of interest and data for a previous campaign in relation to a population group selected for the previous campaign;
- performing, by the computing device before commencement of the campaign of interest, steps including:
- generating a trained lift model to predict one or more effects of the previous campaign with respect to at least one key performance indicator (KPI) among the population group, generating the trained lift model comprising:
- collecting attribute data of a first group of individuals who were reached by the previous campaign and a second group of individuals who were not reached by the previous campaign;
- creating (i) a first training set comprising the collected attribute data of the first group and (ii) a second training set comprising the collected attribute data of the second group:
- training a treatment model by applying a first artificial intelligence (AI) algorithm on the first training set to generate a trained treatment model that predicts the at least one KPI for the first group:
- training a control model by applying a second AI algorithm on the second training set to generated a trained control model that predicts the at least one KPI for the second group;
- and
- combining the trained treatment model and the trained control model to generate the trained lift model;
- ranking the population group based on the trained lift model to generate an initial ranking;
- and
- selecting, from the population group, individuals to contact by the campaign of interest based on the initial ranking;
- and
- calculating, by the computing device, the surrogate ground truth after completion of the campaign of interest that targets the individuals selected from the population group based on the initial ranking, calculating the surrogate truth comprising:
- determining actual KPI scores for individuals in a first subgroup that were reached by the campaign of interest, wherein the individuals in the first subgroup are from the individuals selected based on the initial ranking;
- scoring the first subgroup with the trained control model of the lift model to generate estimated KPI scores for the individuals in the first subgroup, each estimated KPI score representing an estimated KPI that the corresponding individual would generate had the individual not been reached by the campaign of interest;
- determining lift scores for the individuals in the first subgroup by subtracting the estimated KPI scores from the actual KPI scores of the first subgroup;
- determining actual KPI scores for individuals in a second subgroup that were not reached by the campaign of interest, wherein the individuals in the second group are from the individuals selected based on the initial ranking;
- scoring the second subgroup with the trained treatment model of the lift model to generate estimated KPI scores for the individuals in the second subgroup, each score representing an estimated KPI that the corresponding individual would generate had the individual been reached by the campaign of interest;
- determining lift scores for the individuals in the second subgroup by subtracting the actual KPI scores from the estimated KPI scores of the second subgroup;
- and
- calculating the surrogate ground truth for the campaign of interest based on the lift scores for the first and second subgroups.