Orthogonal experimentation in a computing environment
AssigneeeBay Inc.
InventorsJasdeep Singh Sahni, Anil Madan, Deepak Seetharam Nadig, Po Cheung, Bhavesh Mistry, John Bodine, Michael Lo
Life of this patent
Estimating the term…
Abstract
Various embodiments include at least one of systems, methods, and software to receive input configuring tests within a computing environment to expose users to standard application or website experiences or test experiences. In some embodiments, multiple tests may be configured to run orthogonally within user experiences without affecting the results of one another. Some such embodiments preserve the ability to execute certain tests in a non-orthogonal manner while other tests are allowed to execute orthogonally.
Claims20
A method comprising:
- receiving a first experiment identifier for a first experiment;
- receiving a first traffic configuration setting for the first experiment, the first traffic configuration setting indicating a portion of traffic allotted to the first experiment;
- receiving a first traffic allotment for a first experiment variant of the first experiment and a second traffic allotment for a second experiment variant of the first experiment;
- receiving a field value indicating that traffic to the first experiment is mutually excluded from other traffic to a second experiment;
- and
- storing the first experiment identifier, the first traffic configuration setting, the first traffic allotment, the second traffic allotment in an experiment configuration file.
A system comprising:
- at least one processor;
- and
- a memory device storing instructions that, when executed by the at least one processor, cause the system perform operations comprising:
- receiving a first experiment identifier for a first experiment;
- receiving a first traffic configuration setting for the first experiment, the first traffic configuration setting indicating a portion of traffic allotted to the first experiment;
- receiving a first traffic allotment for a first experiment variant of the first experiment and a second traffic allotment for a second experiment variant of the first experiment;
- receiving a field value indicating that traffic to the first experiment is mutually excluded from other traffic to a second experiment;
- and
- storing the first experiment identifier, the first traffic configuration setting, the first traffic allotment, the second traffic allotment in an experiment configuration file.
The system of clam 9, wherein the operations further comprise assigning a value to the user corresponding to the user identifier.
The system of clam 8, wherein the operations further comprise receiving second data indicating that traffic to the second experiment is not mutually excluded from traffic to a third experiment.
The system of clam 8, wherein the operations further comprise generating a user interface comprising at least one field for input of the field value indicating that traffic to the first experiment is mutually excluded from other traffic to a second experiment,
A non-transitory computer-readable storage medium, with instructions stored thereon, which when executed by at least one processor of a computer, causes the computer to perform operations comprising:
- receiving a first experiment identifier for a first experiment;
- receiving a first traffic configuration setting for the first experiment, the first traffic configuration setting indicating a portion of traffic allotted to the first experiment;
- receiving a first traffic allotment for a first experiment variant of the first experiment and a second traffic allotment for a second experiment variant of the first experiment;
- receiving a field value indicating that traffic to the first experiment is mutually excluded from other traffic to a second experiment;
- and
- storing the first experiment identifier, the first traffic configuration setting, the first traffic allotment, the second traffic allotment in an experiment configuration file.
The non-transitory computer-readable storage medium of clam 16, wherein the operations further comprise assigning a value to the user corresponding to the user identifier.
The non-transitory computer-readable storage medium of clam 15, wherein the operations further comprise receiving second data indicating that traffic to the second experiment is not mutually excluded from traffic to a third experiment.
Dependencies are inferred from the claim text.