Offline simulation of multiple experiments with variant adjustments

A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

• identify a plurality of content campaigns for simulated experiments of a plurality of experimental variants, each content campaign having an associated presentation budget;
• for each content campaign in the plurality of content campaigns:
• determine a set of campaign metrics for the content campaign for each of the plurality of experimental variants by presentation of the content campaign with the plurality of experimental variants during an experimental time period until the associated presentation budget for the content campaign is reached, and
• determine a set of adjusted campaign metrics for the content campaign for each of the experimental variants based on the set of campaign metrics and an adjustment based on a portion of the presentation budget used by the experimental variant relative to a fair value of the experimental variant;
• determine a set of simulated experimental results for each experimental variant of the plurality of experimental variants by combining the set of adjusted campaign metrics associated with the experimental variant for each of the plurality of content campaigns;
• and
• based on the set of simulated experimental results, selecting an experimental variant for presentation of content.

A computer system comprising:

• a processor;
• and
• a non-transitory computer readable storage medium storing instructions that, when executed by the processor, cause the computer system to perform actions comprising:
• identifying a plurality of content campaigns for experiments of a plurality of experimental variants, each content campaign having an associated presentation budget;
• for each content campaign in the plurality of content campaigns:
• determining a set of campaign metrics for the content campaign for each of the plurality of experimental variants by presentation of the content campaign with the plurality of experimental variants during an experimental time period until the associated presentation budget for the content campaign is reached, and
• determining a set of adjusted campaign metrics for the content campaign for each of the experimental variants based on the set of campaign metrics and an adjustment based on a portion of the presentation budget used by the experimental variant relative to a fair value of the experimental variant;
• determining a set of simulated experimental results for each experimental variant of the plurality of experimental variants by combining the set of adjusted campaign metrics associated with the experimental variant for each of the plurality of content campaigns and
• based on the set of simulated experimental results, selecting an experimental variant for presentation of content.