Delayed processing for over-delivery determination for content delivery system experimentation

A system comprising:

• one or more processors;
• storage media;
• and
• instructions stored in the storage media and which, when executed by the one or more processors, cause the system to perform:
• receiving, during a batch window, a set of campaign data for content item campaign deliveries made during an auction experiment having a budget-split design or a quota-split design, the set of campaign data containing content from multiple campaigns, including previous campaigns, based on a specified delivery interval where the content was eligible for delivery;
• after the batch window, identifying one or more campaigns that were over-delivered during the auction experiment;
• comparing a pass-through rate for an over-delivered campaign, of the one or more over-delivered campaigns, to a pass-through rate threshold for the over-delivered campaign;
• and
• based on the pass-through rate for the over-delivered campaign being below the pass-through rate threshold for the over-delivered campaign, removing the over-delivered campaign from at least one iteration of the auction experiment or stopping the auction experiment.

One or more non-transitory computer-readable media comprising:

• instructions which, when executed by one or more computing devices of a computing system, cause the computing system to perform:
• receiving a set of campaign data for content item campaign deliveries made during an auction experiment having a budget-split design or a quota-split design, the set of campaign data containing content from multiple campaigns, including previous campaigns, based on a specified delivery interval where the content was eligible for delivery;
• based on the set of campaign data, identifying a campaign that was over-delivered during the auction experiment;
• comparing a pass-through rate for the campaign to a pass-through rate threshold for the campaign;
• and
• based on the pass-through rate for the campaign being below the pass-through rate threshold for the campaign, removing the campaign from the auction experiment.