# US 2017/0372352 A1 · Real-time bidding through placebo-based experimentation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0372352 A1
- Document ID: US20170372352A1
- Status: Published application
- Kind: A1
- Application number: 15/697,346
- Filed: 2017-09-06
- Published: 2017-12-28
- Earliest claimed: 2015-09-10
- Assignees: Adobe Systems Incorporated
- Applicants: Adobe Systems Incorporated
- Inventors: Thomas Riordan; Narayan Kinhal; John Hughes; Jason Lopatecki; Darren Sue; Christopher Bell; Matthew Ellinwood
- Web page: https://patentgrep.com/patent/US20170372352A1

## CPC

- G06Q 30/0244: {Optimization}
- G06Q 30/0246: {Traffic}
- G06Q 30/0275: {Auctions}
- G06Q 30/0277: {Online advertisement}
- G06Q 30/0249: {based upon budgets or funds}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Systems and methods for operating placebo-based experiments are described for online advertisements. One or more embodiments of the disclosed systems and methods utilizes an ad swapping approach to offer placebo media exposures, at no additional cost to an advertiser. One or more embodiments further provide a native experimentation platform that allows users to run tests of ad placements to measure the effectiveness of ads and view results displayed on a user interface. The disclosed systems and methods can assign viewers into a test group if shown the test ad or a control group if shown a control ad. The control ad can be provided at no cost to the advertiser for embodiments where the placebo ad belongs to an alternative advertiser. Effectiveness of third party attribution can also be evaluated. The disclosed systems and methods can define experiment parameters, including control frequency, test viewer groups, and control viewer groups.

## Claims

### Claim 1

In a digital medium environment for real-time bidding on impression opportunities associated with viewers simultaneously accessing digital content, a computer-implemented method of conducting a placebo-based experiment for improving online campaigns, comprising:
identifying a first set of campaign parameters of a first advertiser and a second set of campaign parameters of a second advertiser;
running a media lift experiment for the first advertiser by:
receiving, from a remote server, a bid request package for an impression opportunity associated with a first viewer visiting a website;
determining, by at least one processor, that the first viewer corresponds to a control group; and
based on the first set of campaign parameters, the second set of campaign parameters, and the determination that the first viewer corresponds to the control group, automatically responding, by the at least one processor, to the bid request package by sending a control ad associated with the second advertiser to the remote server for display to the first viewer via the website; and
determining a measure of media lift for a test ad associated with the first advertiser based on performance of the control ad associated with the second advertiser.

### Claim 2 (depends on 1)

The method of claim 1, wherein automatically responding to the bid request package comprises sending a bid for the control ad associated with the second advertiser to the remote server within 250 milliseconds of receiving the bid request package from the remote server.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
automatically modifying the first set of campaign parameters based on the measure of media lift; and
running an online ad campaign with the modified first set of campaign parameters by:
receiving, from the remote server, a second bid request package for a second impression opportunity associated with a second viewer; and
responding to the second bid request package based on the modified first set of campaign parameters.

### Claim 4 (depends on 3)

The method of claim 3, wherein automatically modifying the first set of campaign parameters comprises at least one of: modifying viewer targeting parameters or modifying a budget for one or more advertisements.

### Claim 5 (depends on 1)

The method of claim 1, wherein running the media lift experiment for the first advertiser further comprises:
determining that the first viewer satisfies both the first set of campaign parameters of the first advertiser and the second set of campaign parameters of the second advertiser; and
automatically responding to the bid request package by sending the control ad associated with the second advertiser based on the determination that the first viewer satisfies both the first set of campaign parameters and the second set of campaign parameters.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
receiving, from the remote server, a second bid request package for a second impression opportunity associated with a second viewer;
determining that the second viewer corresponds to a test group; and
based on the first set of campaign parameters and the determination that the first viewer corresponds to the test group, automatically responding to the second bid request package by sending the test ad associated with the first advertiser to the remote server for display to the second viewer.

### Claim 7 (depends on 6)

The method of claim 6, wherein determining the measure of media lift comprises:
identifying a first user interaction by the first viewer with the control ad associated with the second advertiser;
identifying a second user interaction by the second viewer with the test ad associated with the first advertiser; and
comparing the first user interaction by the first viewer with the control ad associated with the second advertiser with the second user interaction by the second viewer with the test ad associated with the first advertiser.

### Claim 8 (depends on 7)

The method of claim 7, wherein:
determining that the first viewer corresponds to the control group comprises assigning the first viewer to the control group by applying a randomizing operation and control frequency threshold value; and
determining that the second viewer corresponds to the test group comprises assigning the second viewer to the test group by applying the randomizing operation and the control frequency threshold value.

### Claim 9

A system comprising:
at least one server; and
at least one non-transitory computer readable storage medium storing instructions that, when executed by the at least one server, cause the system to:
identify a first set of targeting parameters of a first advertiser and a second set of targeting parameters of a second advertiser;
based on a determination that a first viewer satisfies the first set of targeting parameters and the second set of targeting parameters and based on a determination that the first viewer corresponds to a control group, automatically sending a control ad associated with the second advertiser for display to the first viewer; and
determine a measure of media lift for a test ad associated the first advertiser based on performance of the control ad associated with the second advertiser.

### Claim 10 (depends on 9)

The system of claim 9, wherein automatically sending the control ad comprises automatically sending a bid for the control ad associated with the second advertiser within 250 milliseconds of receiving a bid request package.

### Claim 11 (depends on 9)

The system of claim 9, further comprising instructions that, when executed by the at least one server, cause the system to:
automatically modify the first set of targeting parameters based on the measure of media lift; and
run an online ad campaign with the modified first set of targeting parameters by:
receiving a bid request package for an impression opportunity associated with a second viewer; and
responding to the bid request package for the impression opportunity associated with the second viewer based on the modified first set of targeting parameters.

### Claim 12 (depends on 9)

The system of claim 9, further comprising instructions that, when executed by the at least one server, cause the system to:
receive a bid request package for an impression opportunity associated with a second viewer;
determine that the second viewer corresponds to a test group; and
based on the first set of targeting parameters and the determination that the first viewer corresponds to the test group, automatically responding to the bid request package by sending the test ad associated with the first advertiser for display to the second viewer.

### Claim 13 (depends on 12)

The system of claim 12, further comprising instructions that, when executed by the at least one server, cause the system to determine the measure of media lift by:
identifying a first user interaction by the first viewer with the control ad associated with the second advertiser;
identifying a second user interaction by the second viewer with the test ad associated with the first advertiser; and
comparing the first user interaction by the first viewer with the control ad associated with the second advertiser with the second user interaction by the second viewer with the test ad associated with the first advertiser.

### Claim 14 (depends on 9)

The system of claim 9, further comprising instructions that, when executed by the at least one server, cause the system to:
receive a bid request package for an impression opportunity associated with a second viewer;
determine that the second viewer corresponds to a control group of the second advertiser; and
based on a determination the second viewer satisfies the first set of targeting parameters and the second set of targeting parameters and based on the determination that the second viewer corresponds to the control group of the second advertiser, automatically send a control ad associated with the first advertiser for display to the second viewer; and
determine a measure of media lift for a test ad associated the second advertiser based on performance of the control ad associated with the first advertiser.

### Claim 15

In a digital medium environment for real-time bidding on impression opportunities associated with viewers simultaneously accessing digital content, a computer-implemented method of conducting a placebo-based experiment for improving online campaigns, comprising:
receiving a bid request package for an impression opportunity associated with a first viewer;
running a media lift experiment for a first advertiser by sending a control ad associated with a second advertiser for display to the first viewer;
comparing performance of the control ad associated with the second advertiser to a test ad associated with the first advertiser; and
modifying one or more campaign parameters of the first advertiser based on the comparison between the performance of the control ad associated with the second advertiser and the test ad associated with the first advertiser.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
determining that the first viewer corresponds to a control group of the first advertiser; and
sending the control ad associated with the second advertiser based on the determination the first viewer corresponds to the control group.

### Claim 17 (depends on 16)

The method of claim 16, further comprising:
identifying the one or more campaign parameters of the first advertiser and one or more campaign parameters of the second advertiser, wherein the one or more campaign parameters of the first advertiser comprise a first set of targeting parameters and the one or more campaign parameters of the second advertiser comprise a second set of targeting parameters; and
sending the control ad associated with the second advertiser based on a determination that the first viewer satisfies the first set of targeting parameters and the second set of targeting parameters.

### Claim 18 (depends on 17)

The method of claim 17, wherein modifying the one or more campaign parameters of the first advertiser based on the comparison comprises modifying the first set of targeting parameters based on the comparison and further comprising:
running an online ad campaign with the modified first set of targeting parameters by:
receiving a second bid request package for a second impression opportunity associated with a second viewer; and
responding to the second bid request package based on the modified first set of targeting parameters.

### Claim 19 (depends on 15)

The method of claim 15, further comprising:
receiving a second bid request package for a second impression opportunity associated with a second viewer;
determining that the second viewer corresponds t
o a test group of the first advertiser; and
based on the first set of targeting parameters and the determination that the first viewer corresponds to the test group of the first advertiser, automatically responding to the second bid request package by sending the test ad associated with the first advertiser for display to the second viewer.

### Claim 20 (depends on 19)

The method of claim 19, wherein modifying the one or more campaign parameters of the first advertiser comprises:
identifying a first user interaction by the first viewer with the control ad associated with the second advertiser;
identifying a second user interaction by the second viewer with the test ad associated with the first advertiser; and
comparing the first user interaction by the first viewer with the control ad associated with the second advertiser with the second user interaction by the second viewer with the test ad associated with the first advertiser.
