Systems, methods, and media for comparing variations of electronic content in relation to a measured metric and/or generating a causal model to calculate the effect of an input variable on a desired outcome

A system for performing A/B testing for a first variation of electronic web-based content and a second variation of the electronic web-based content, the system comprising:

• a memory;
• a processor coupled to the memory, the processor executing a software module configured to:
• receive, from a client device operated by a user, A/B testing input information for the first variation of the electronic web-based content and the second variation of the electronic web-based content, wherein the first variation of the electronic web-based content is utilized by a first group of a plurality of first users for the A/B testing and the second variation of the electronic web-based content is utilized by a second group of a plurality of second users for the A/B testing;
• determine, based on the A/B testing input information, a type of a variable corresponding to a feature that is different in the first variation of the electronic web-based content and the second variation of the electronic web-based content;
• determine a distribution of a performance metric among the plurality of first users and among the plurality of second users, wherein the performance metric is related to the variable corresponding to the feature;
• identify a statistical test based on the determined distribution among the plurality of first users and the plurality of second users;
• compute, for the statistical test, a first probability value for a false positive in performance of the A/B testing;
• compute, for the statistical test, a second probability value for a false negative in performance of the A/B testing;
• subtract, for the statistical test, the second probability value from 1 to produce a statistical power for the A/B testing;
• compute a first mean value for the distribution of the performance metric among the plurality of first users and a second mean value for the distribution of the performance metric among the plurality of second users;
• define a difference in the performance metric across the first group utilizing the first variation and the second group utilizing the second variation as significant when (1) the first probability is less than a first threshold value, (2) the difference between the first mean value and the second mean value is greater than a second threshold value, and (3) the statistical power is greater than a third threshold value;
• and
• detect that (1) the first probability is less than the first threshold value, (2) the difference between the first mean value and the second mean value is greater than the second threshold value, and (3) the statistical power is greater than a third threshold value;
• and
• automatically update, in response to the detecting, a web page to include the web-based electronic content with the feature.

A computer-implemented method for performing a measurement for a set of a plurality of different interventions or values used for obtaining a desired outcome related to electronic content, the method comprising:

• obtaining first information indicating whether each user of a plurality of users received and/or interacted with each of the plurality of different interventions;
• obtaining transaction history information for each of the plurality of users;
• identifying one or more factors indicating whether each user, of the plurality of different users, is eligible for each of the plurality of different interventions or values;
• identifying one or more market conditions for each of the plurality of different interventions or values;
• generating a causal model including a plurality of nodes and directed edges, wherein the causal model includes a different first node for representing each of the first information, transaction history, one or more factors, one or more market conditions, and each of the plurality of different interventions, and a second node for representing the desired outcome, and wherein there is at least one directed edge from each of the different first nodes to the second node;
• fitting a regression model to each particular node to determine the causal impact on the particular node by other nodes of the causal model using the directed edges;
• calculating an impact value for each of the plurality of different interventions or values based on fitting the regression model to each particular node;
• and
• measuring a variable impact value for an identified variable corresponding to the plurality of different interventions or values.