Online testing efficiency through early termination

A system for presenting a termination notification of an experiment comparing a first version of a content page with a second version of the content page through a user interface, the system comprising:

• one or more processors;
• and
• a memory that stores computer-executable instructions, wherein the computer-executable instructions, when executed, cause the one or more processors to:
• obtain first data during a first time period of execution of the experiment;
• determine, based on the first data, a predicted statistical result of the experiment;
• determine, based on the predicted statistical result, that a predicted duration of the experiment includes the first time period and subsequent time periods that end in time after the first time period;
• determine a predicted sample size of each of the first time period and the subsequent time periods;
• generate a first probability value of the experiment that is valid as of the first time period based on at least one of (i) the first data or (ii) the predicted sample size of each of the first time period and the subsequent time periods;
• for a second time period of the subsequent time periods that ends in time after the first time period:
• obtain subsequent data during the second time period;
• generate a second probability value of the experiment that is valid as of the second time period based on at least one of (i) the subsequent data obtained during the second time period, (ii) the predicted sample size of each of the first time period and the subsequent time periods, (iii) the first probability value, or (iv) individual second probability values valid as of each of the subsequent time periods that precede in time the second time period;
• determine that the second probability value of the experiment that is valid as of the second time period satisfies a threshold value;
• and
• in response to determining that the second probability value of the experiment that is valid as of the second time period satisfies the threshold value, cause the user interface to display the termination notification,
• wherein the termination notification, when selected, causes the experiment to terminate prior to completion of the predicted duration.

One or more non-transitory computer-readable media comprising computer-executable instructions for presenting a termination notification of an experiment, wherein the computer-executable instructions, when executed by a computer system, cause the computer system to perform operations comprising:

• obtaining first data during a first time period of execution of the experiment;
• determining that a predicted duration of the experiment includes the first time period and a second time period;
• determining a predicted sample size of each of the first time period and the second time period;
• for the second time period:
• obtaining subsequent data during the second time period;
• generating a second probability value of the experiment that is valid as of the second time period;
• determining that the second probability value of the experiment that is valid as of the second time period satisfies a threshold value;
• and
• in response to determining that the second probability value of the experiment that is valid as of the second time period satisfies the threshold value, cause a user interface to display the termination notification, wherein the termination notification, when selected, causes the experiment to terminate prior to completion of the predicted duration.