US 8,789,019 B2Granted

Orthogonal experimentation in a computing environment

AssigneeeBay Inc.

InventorsJasdeep Singh Sahni, Anil Madan, Deepak Seetharam Nadig, Po Cheung, Bhavesh Mistry, John Bodine, Michael Lo

Life of this patent

Estimating the term…

Loading patent…

Abstract

Various embodiments include at least one of systems, methods, and software to receive input configuring tests within a computing environment to expose users to standard application or website experiences or test experiences. In some embodiments, multiple tests may be configured to run orthogonally within user experiences without affecting the results of one another. Some such embodiments preserve the ability to execute certain tests in a non-orthogonal manner while other tests are allowed to execute orthogonally.

Claims16

Dependent claims

A system comprising:

at least one processor;
at least one memory device;
at least one network interface device;
and
instructions held in the at least one memory device, the instructions retrievable from the at least one memory device and executable by the at least one processor to perform a method comprising:
receiving a selection of a website experiment to be implemented within a computing environment accessible via the at least one network interface device;
receiving input designating the experiment to be implemented individually within a user experience or with an option to coexist with other experiments within a single user experience;
receiving input to provide one or more treatments of the experiment, the input received for each treatment added including:
input identifying the treatment;
and
input identifying a portion of website traffic to which the treatment will be exposed;
storing the received input in an experiment database on a data storage device accessible by computing resources within the computing environment;
receiving, via the at least one network interface device, a request for webpage related content;
performing a hashing function against a user identifier of a user from which the request was received to obtain a hash value;
querying the experiment database as a function of the hash value to identify experiment treatments to which the user is to be exposed;
and
processing the request for the webpage related content according to any identified experiments variants.

14 more elements

A non-transitory computer-readable storage medium, with instructions stored thereon, which when executed by at least one processor of a computer, causes the computer to perform a method comprising:

receiving a selection, within a user interface presented on a display device, of a website experiment to be implemented within a computing environment;
receiving input designating the experiment to be implemented individually within a user experience and with an option to coexist with other experiments within a single user experience;
receiving input to select one or more treatments of the experiment, the input received for each selected treatment including:
input identifying the treatment;
and
input identifying an allotment of website traffic to which the treatment will be exposed;
and
input identifying the allotment of the website traffic to which the treatment will be exposed identifies an allotment of the website traffic to which the experiment is exposed;
based on the received input designating the experiment with an option to coexist with other experiments within a single user experience, receiving input identifying a portion of the website traffic to be exposed to the experiment;
storing the received input in an experiment database on a data storage device accessible by computing resources within the computing environment;
receiving, over a network, a request for webpage related content;
retrieving, from the experiment database, an experiment identifier of each experiment active with regard to the received request;
for each retrieved experiment identifier, performing a hashing function against a combination of a user identifier of a user from which the request was received and the respective experiment identifier to obtain a hash value;
for each experiment identifier and its hash value, comparing the hash value to the portion of the website traffic to be exposed to the experiment of the experiment identifier to determine whether the user is to be exposed to the experiment of the experiment identifier;
for each experiment to which the user is to be exposed, identifying a treatment of the experiment to which the user is to be exposed as a function of the hash value and the allotments of website traffic to the treatments of the experiment;
for each identified experiment to which the user is to be exposed, providing the identified treatment of the experiment, otherwise providing a standard experience;
and
processing the request for the webpage related content according to any identified experiment treatments.

15 more elements

Dependencies are inferred from the claim text.

Facts

Application: 13/867,203Filed 22 Apr 2013 · division
Priority: 19 Mar 2010
Published: US 2013/0239091 A112 Sep 2013
Granted: 22 Jul 2014
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

G06F 11/3668
Prevention of errors by analysis, debugging or testing of software: testing of software
G06F 11/3688
Test management: for test execution, e.g. scheduling of test suites
G06F 11/3696
Testing of software: methods or tools to render software testable
G06F 16/2255
Indexing structures: hash tables

Family

Application12/728,067Parent of this division · filed 19 Mar 2010
US 2013/0239091 A1Pre-grant publication · published 12 Sep 2013
US 8,429,616 B2Parent of this division
US 2011/0231821 A1Published application
US 8,789,019 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key