Determining winning arms of A/B electronic communication testing for a metric using historical data and histogram-based bayesian inference

A system configured to select a winner of an A/B test, comprising:

• a plurality of computing devices;
• a merchant server;
• a server, the server connected to the merchant server and the computing devices through a network, the server configured to:
• obtain historical data related to the A/B test of a merchant of the merchant server;
• create a histogram based on the historical data, the histogram including bins and weights;
• create a Dirichlet distribution by computing concentration parameters of the Dirichlet distribution from the weights of the histogram;
• execute the A/B test with an associated metric comprising testing at least two arms of the A/B test with a user profile;
• receive new data from the plurality of computing devices collected based on recipient actions during the execution of the A/B test and the associated metric;
• allocate the new data into same bins as the bins of the histogram of the historical data yielding a new data bin count for each bin of the allocated new data;
• compute a posterior Dirichlet distribution comprising updating the Dirichlet distribution using the same bins and the new data bin counts and the concentration parameters of the Dirichlet distribution in closed form vie conjugate prior formulae;
• infer corresponding central tendencies of samplings of a metric distribution, wherein each central tendency of the corresponding central tendencies is determined by sampling the posterior Dirichlet distribution;
• construct an overall utility distribution for each arms of the A/B test by combining the central tendency of each sampling of the metric distribution with a corresponding sampling of a conversion probability distribution;
• determine a winning arm of the A/B testing by comparing the overall utility distribution of each arm with each other arm of the A/B test;
• and
• electronically communicate with recipients of the computing devices using a template based on the winning arm of the A/B testing.