US 2016/0253683 A1 · SAMPLING OF USERS IN NETWORK A/B TESTING

Abstract

The disclosed embodiments provide a system for performing network A/B testing. During operation, the system obtains a graph of a social network and calculates a set of equally sized clusters of users in the social network by iteratively switching memberships of the nodes among the equally sized clusters to increase a number of edges in each of the equally sized clusters. Next, the system randomly selects a subset of the equally sized clusters for exposure to a treatment version of a message. The system then performs an A/B test by presenting the treatment version to the selected clusters and tracking a response of the selected clusters to the treatment version.

Claims20

Dependent claims

A method, comprising:

obtaining a graph of a social network, wherein the graph comprises:
a set of nodes representing a set of users;
and
a set of edges representing relationships between pairs of the nodes;
calculating, by one or more computer systems, a set of equally sized clusters of the users in the social network by iteratively switching memberships of the nodes among the equally sized clusters to increase a number of edges in each of the equally sized clusters;
randomly selecting a subset of the equally sized clusters for exposure to a treatment version of a message during an A/B test;
and
performing, by the one or more computer systems, the A/B test by presenting the treatment version to the selected clusters and tracking a response of the selected clusters to the treatment version.

5 more elements

An apparatus, comprising:

one or more processors;
and
memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
obtain a graph of a social network, wherein the graph comprises:
a set of nodes representing a set of users;
and
a set of edges representing relationships between pairs of the nodes;
calculate a set of equally sized clusters of the users in the social network by iteratively switching memberships of the nodes among the equally sized clusters to increase a number of edges in each of the equally sized clusters;
randomly select a subset of the equally sized clusters for exposure to a treatment version of a message during an A/B test;
and
perform the A/B test by presenting the treatment version to the selected clusters and tracking a response to the treatment version from the selected clusters.

8 more elements

A system, comprising:

a sampling non-transitory computer readable medium comprising instructions that, when executed by one or more processors, cause the system to:
obtain a graph of a social network, wherein the graph comprises:
a set of nodes comprising a set of users;
and
a set of edges representing relationships between pairs of the nodes;
use the graph to calculate a set of equally sized clusters of the users in the social network by iteratively switching memberships of the nodes among the equally sized clusters to increase a number of edges in each of the equally sized clusters;
and
randomly selecting one or more clusters from the set of equally sized clusters for exposure to a treatment version of a message during an A/B test;
and
an estimation non-transitory computer readable medium comprising instructions that, when executed by the one or more processors, cause the system to:
perform the A/B test by presenting the treatment version to the selected clusters and tracking a response to the treatment version from the selected clusters;
select a fraction of additional users in the social network for subsequent exposure to the treatment version by analyzing the response to the treatment version;
and
present the treatment version to the fraction of additional users.

11 more elements

Dependencies are inferred from the claim text.

Sampling of users in network a/b testing

Life of this patent

Sampling of users in network a/b testing

Life of this patent

Abstract

Claims20