# US 2017/0286997 A1 · Advertisement conversion prediction based on unlabeled data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286997 A1
- Document ID: US20170286997A1
- Status: Published application
- Kind: A1
- Application number: 15/091,105
- Filed: 2016-04-05
- Published: 2017-10-05
- Earliest claimed: 2016-04-05
- Assignees: Facebook, Inc.
- Applicants: Facebook, Inc.
- Inventors: Sameer Indarapu; Pradheep K. Elango; Xian Xu
- Web page: https://patentgrep.com/patent/US20170286997A1

## CPC

- G06Q 30/0247: {Calculate past, present or future revenues}
- G06N 20/00: Machine learning
- G06Q 30/0277: {Online advertisement}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Embodiments are disclosed for predicting target events occurrence for an advertisement campaign. A computing device according to some embodiments assigns a label to an advertisement as unlabeled, in response to a notification that a prerequisite event occurs for the advertisement. The device generates feature vectors based on data that relate to the advertisement. The device further trains a machine learning model using the feature vectors of the unlabeled advertisement based on a first term of an objective function, without waiting for a target event for the advertisement to occur. The first term depends on unlabeled advertisements. The device predicts a probability of a target event occurring for a new advertisement, by feeding data of the new advertisement to the trained machine learning model.

## Claims

### Claim 1

A method for predicting target events occurrence for an advertisement campaign, comprising:
assigning, by a computing device, a label to an advertisement as unlabeled, in response to a notification that a prerequisite event occurs for the advertisement;
generating feature vectors based on data that relate to the advertisement;
training, by the computing device, a machine learning model using the feature vectors of the unlabeled advertisement based on a first term of an objective function, the first term depending on unlabeled advertisements, without waiting for a target event for the advertisement to occur; and
predicting a probability of a target event occurring for a new advertisement, by feeding data of the new advertisement to the trained machine learning model.

### Claim 2 (depends on 2)

The method of claim 2, further comprising:
receiving, by the computing device, a second notification that a target event occurs for the advertisement;
in response to the second notification, changing the label of the advertisement from unlabeled to positive;
further training, by the computing device, the machine learning model using the feature vectors of the positive advertisement based on a second term of an objective function, the second term depending on positive advertisements.

### Claim 3 (depends on 1)

The method of claim 1, wherein the objective function is a logarithmic loss function including two terms, the first term depends on numerical values representing positive advertisements of which target events have occurred, and the second term depends on numerical values representing unlabeled advertisements of which prerequisite events have occurred but target events have not occurred.

### Claim 4 (depends on 3)

The method of claim 3, wherein the objective function includes no term depending on numerical values representing negative instances of the advertisement of which target events never occurred.

### Claim 5 (depends on 1)

The method of claim 1, wherein the prerequisite event includes a clicking event when a web link included in the advertisement is clicked, and the target event includes a conversion event that relates to the advertisement.

### Claim 6 (depends on 1)

The method of claim 1, wherein the conversion event is an event of a purchase transaction or a subscription as a result of the advertisement.

### Claim 7 (depends on 1)

The method of claim 1, wherein the prerequisite event includes an impression event when the advertisement is rendered on a device screen for a user, and the target event includes a clicking event when a web link included in the advertisement is clicked.

### Claim 8 (depends on 1)

The method of claim 1, wherein the step of predicting comprises:
predicting the probability of the target event occurring for the new advertisement, by the machine learning model, without a delay of a predetermined time window, wherein if no target event occurs within the predetermined time window, the instance of the advertisement is treated as negative advertisement of which the target event never occur as a result of the advertisement.

### Claim 9 (depends on 1)

The method of claim 1, wherein the step of predicting comprises:
predicting the probability of a target event occurring for the new advertisement, by the machine learning model, before or within a lifetime of the advertisement campaign when instances of the advertisement are distributed.

### Claim 10 (depends on 2)

The method of claim 2, wherein the step of further training comprises:
dynamically training the machine learning model using the feature vectors of the positive advertisement in real time when the target event occurs for the advertisement; and
continuing to predict probabilities of target events occurring for advertisements when the machine learning model is trained dynamically.

### Claim 11 (depends on 1)

The method of claim 1, wherein the step of training comprises:
adjusting parameters of the machine learning model to minimize the objective function.

### Claim 12 (depends on 1)

The method of claim 1, wherein the step of training comprising;
adjusting parameters of the machine learning model to minimize the objective function through a stochastic gradient descent process.

### Claim 13 (depends on 1)

The method of claim 1, wherein the feature vectors further includes vectors based on data of a user profile for a user who incurs the prerequisite event as a result of the advertisement.

### Claim 14 (depends on 1)

The method of claim 1, wherein the first term of the objective function is proportionate to
∑
i
=
1
N


[
V
i
pre

log


(
1
-
P
i
)
]
,
wherein for each advertisement i, Vipre is a numerical value representing an unlabeled advertisement of which a prerequisite event has occurred but no target event has occurred, Pi is the probability that a target event occurs as a result of the advertisement i; and
wherein the second term of the objective function is proportionate to
∑
i
=
1
N


[
V
i
+

log

(
P
i
1
-
P
i
)
]
,
wherein for each advertisement i, Vi+ is a numerical value representing a positive instance of which a target event has occurred.

### Claim 15

A non-transitory computer-readable storage medium storing instructions for providing a prediction whether conversions occur for advertisements, the instructions comprising:
receiving a message indicating that a clicking event occurs for an advertisement rendered on a user computer device;
in response to the message, assigning a label to the instance of the advertisement as unlabeled;
generating a set of feature vectors based on data of the advertisement;
training a machine learning model using the set of feature vectors of the unlabeled instance based on a first training algorithm for the unlabeled advertisement; and
generating, by the machine learning model, a prediction of probability on whether a conversion event occur after a clicking event for a new advertisement, by feeding data of the new advertisement to the machine learning model.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein the instructions further comprise:
providing, based on the prediction of probability on whether a conversion event occur after a clicking event for the new advertisement, a recommended price for distributing the advertisement on a cost-per-conversion basis.

### Claim 17 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein the instructions further comprise:
receiving a second message indicating that a conversion event occurs after the clicking event as a result of the advertisement;
in response to the second message, changing the label of the advertisement from unlabeled to positive; and
updating the machine learning model using the set of feature vectors of the positive advertisement based on a second training algorithm for positive advertisements.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the first training algorithm is based on a first term of an objective function that depends on numerical values representing positive advertisements and probabilities that conversion events occur for the advertisement; wherein the second training algorithm is based on a second term of the objective function that depends on numerical values representing unlabeled advertisements.

### Claim 19

A computing system, comprising:
a memory for storing parameters of a machine learning model for predicting whether clicking events occur for an advertisement;
a labeling module configured to label the advertisement as unlabeled, in response to an impression event occurs when the advertisement is rendered on a user device;
a feature extraction module configured to generate one or more feature vectors based on data of the unlabeled advertisement; and
a training module configured to train the parameters of the machine learning model, without waiting for a clicking event as a result of the advertisement, using the feature vectors of the unlabeled advertisement, by minimizing an objective function including terms for unlabeled advertisements of which impression events have occurred but no clicking events have occurred.

### Claim 20 (depends on 19)

The computing system of claim 19, further comprising:
a prediction module configured to provide a probability prediction on whether a clicking event will occur for a new advertisement, by feeding feature vectors of the new advertisement to the machine learning model.

### Claim 21 (depends on 20)

The computing system of claim 20, further comprising:
a pricing module configured to provide a recommended price for distributing the new advertisement on a cost-per-click basis, based on the probability prediction on whether a clicking event will occur for the new advertisement.

### Claim 22 (depends on 19)

The computing system of claim 19, wherein the labeling module is further configured to label the advertisement as positive, in response to a clicking event occurs as a result of the advertisement, and
wherein the training module is further configured to update the parameters of the machine learning model, by minimizing the objective function including terms for positive advertisements of which clicking events have occurred.