US 2025/0292284 A1 · System and Method to Measure the Performance of an Online Advertisement and Update the Control Group in Real Time · patentgrep

Loading patent…

Abstract

A measurement platform configured to measure performance of an advertisement, the measurement platform comprising a demand-side platform (DSP), the measurement platform configured to send to an end user a survey configured to measure the advertisement's performance on behalf of a client DSP, wherein the measurement platform is further configured to update an exposed group of exposed users, wherein the measurement platform is further configured to update a control group of end users with similar statistical properties to the exposed group except that the control users have not seen the winning advertisement, wherein the measurement platform is further configured to send the survey to the end user, wherein the measurement platform is further configured to receive a survey response from the end user, wherein the measurement platform determines a metric measuring performance of the winning advertisement, wherein the measurement platform is further configured to send the metric to the client DSP.

Claims63

Dependent claims

A measurement platform configured to measure performance of an advertisement, the measurement platform comprising a demand-side platform (DSP), the measurement platform configured to send to an end user a survey configured to measure the advertisement's performance on behalf of a client DSP, the client DSP comprising a DSP of a client of the measurement platform, wherein the measurement platform is further configured to receive an advertisement display notification, the advertisement display notification notifying the measurement platform that a device under control of the end user (“user device”) has displayed to the end user a winning advertisement from the client DSP that wins an auction run by a supply-side platform (SSP) operably connected to the DSP, the advertisement display notification comprising an identifier of the end user, wherein on receipt of the advertisement display notification, the measurement platform stores the identifier of the exposed end user (“exposed user identifier”) in an exposed group of end users (“exposed users”) that have seen the winning advertisement, wherein the measurement platform is further configured to update an exposed group of the exposed users, wherein the measurement platform is further configured to update a control group of end users with similar statistical properties to the exposed group except that the control group members (“control users”) have not seen the winning advertisement, wherein the measurement platform is further configured to send the survey to the end user, wherein the measurement platform is further configured to receive a response to the survey from the end user, wherein the measurement platform is further configured to determine a metric measuring performance of the winning advertisement, wherein the measurement platform is further configured to send the metric to the client DSP.

A method for measuring effectiveness of an advertisement using a measurement platform, the method comprising:

receiving, by the measurement platform, the measurement platform configured to measure performance of an advertisement, the measurement platform comprising a demand-side platform (DSP), the measurement platform further comprising a profile storage configured to store an end user profile, the end user profile comprising a set of features associated with the end user, the measurement platform further comprising a control group storage configured to store an identifier of a control user (“control user identifier”), the measurement platform further comprising a control group builder operably connected to the data platform, the control group builder configured to update the control group with similar statistical properties to an exposed group of end users (“exposed users”) that have seen the winning advertisement, an advertisement display notification notifying the measurement platform that a device under control of the end user (“user device”) has displayed to the end user a winning advertisement from a client DSP, the client DSP comprising a DSP of a client of the measurement platform, the winning advertisement having won an auction run by a supply-side platform (SSP) operably connected to the DSP, the advertisement display notification comprising an identifier of the end user;
storing, by the measurement platform, the identifier of the exposed end user (“exposed user identifier”) in the exposed group;
updating, by the measurement platform, the exposed group;
updating, by the control group builder, with a new control user identifier, a control group of end users with similar statistical properties to the exposed group except that the control group members (“control users”) have not seen the winning advertisement;
storing, by the control group builder, the identifier of the new control user (“control user identifier”) in the control group storage;
receiving, by the measurement platform, an advertisement request copy from the SSP;
sending, by the measurement platform, to the end user, a survey configured to measure performance of the advertisement;
receiving, by the measurement platform, a response to the survey from the end user;
determining, by the measurement platform, a metric measuring performance of the winning advertisement;
and
sending, by the measurement platform, the metric to the client DSP.

8 more elements

A method for measuring effectiveness of an advertisement using a measurement platform, the method comprising:

receiving, by the measurement platform, the measurement platform configured to measure performance of an advertisement, the measurement platform comprising a demand-side platform (DSP), the measurement platform further comprising a profile storage configured to store an end user profile, the end user profile comprising a set of features associated with the end user, the measurement platform further comprising a control group storage configured to store an identifier of a control user (“control user identifier”), the measurement platform further comprising a control group builder operably connected to the data platform, the control group builder configured to update the control group with similar statistical properties to an exposed group of end users (“exposed users”) that have seen the winning advertisement, an advertisement display notification notifying the measurement platform that a device under control of the end user (“user device”) has displayed to the end user a winning advertisement from a client DSP, the client DSP comprising a DSP of a client of the measurement platform, the winning advertisement having won an auction run by a supply-side platform (SSP) operably connected to the DSP, the advertisement display notification comprising an identifier of the end user;
storing, by the measurement platform, the identifier of the exposed end user (“exposed user identifier”) in the exposed group;
updating, by the exposed group storage, the exposed group;
retrieving, by the control group builder, a selected portion of the exposed user identifiers;
retrieving, by the control group builder, a selected portion of the control user identifiers;
retrieving, by the control group builder, a selected portion of the end user profiles comprised in the profile storage;
encoding, by the control group builder, an end user profile into a binary feature vector, wherein at least one element of the binary feature vector equals 0 or 1;
sorting, by the control group builder, the binary feature vector into one of a group comprising exposed binary feature vectors (“an exposed binary feature vector group”), a group comprising control binary feature vectors (“a control binary feature vector group”), and a control pool comprising all available binary feature vectors that are not comprised in one or more of the exposed binary feature vector group and the control binary feature vector group;
determining, by the control group builder, using the exposed binary feature vector group and using the control binary feature vector group, an actual index vector, the actual index vector comprising a measure of a difference between the binary feature vectors corresponding to the exposed users (“exposed binary feature vectors”) and the binary feature vectors corresponding to the control users (“control binary feature vectors”);
determining, by the control group builder, a dot product between the actual index vector and a binary feature vector comprised in the control pool;
storing, by the control group builder, the dot product in the control group storage;
selecting, by the control group builder, a best control binary feature vector having a maximum value of the dot product;
transferring, by the control group builder, the best control binary feature vector from the control pool to the control group;
storing, by the control group builder, the identifier of the new control user (“control user identifier”) in the control group storage;
determining, by the measurement platform, that a size of the control group does not comprise at least a required control group size;
and
returning, by the measurement platform, to the step of retrieving all available exposed user identifiers.

14 more elements

A method for measuring effectiveness of an advertisement using a measurement platform, the method comprising:

receiving, by the measurement platform, the measurement platform configured to measure performance of an advertisement, the measurement platform comprising a demand-side platform (DSP), the measurement platform further comprising a profile storage configured to store an end user profile, the end user profile comprising a set of features associated with the end user, the measurement platform further comprising a control group storage configured to store an identifier of a control user (“control user identifier”), the measurement platform further comprising a control group builder operably connected to the data platform, the control group builder configured to update the control group with similar statistical properties to an exposed group of end users (“exposed users”) that have seen the winning advertisement, an advertisement display notification notifying the measurement platform that a device under control of the end user (“user device”) has displayed to the end user a winning advertisement from a client DSP, the client DSP comprising a DSP of a client of the measurement platform, the winning advertisement having won an auction run by a supply-side platform (SSP) operably connected to the DSP, the advertisement display notification comprising an identifier of the end user;
storing, by the measurement platform, the identifier of the exposed end user (“exposed user identifier”) in the exposed group;
updating, by the exposed group storage, the exposed group;
retrieving, by the control group builder, a selected portion of the exposed user identifiers;
retrieving, by the control group builder, a selected portion of the control user identifiers;
retrieving, by the control group builder, a selected portion of the end user profiles comprised in the profile storage;
encoding, by the control group builder, an end user profile into a binary feature vector, wherein at least one element of the binary feature vector equals 0 or 1;
sorting, by the control group builder, the binary feature vector into one of a group comprising exposed binary feature vectors (“an exposed binary feature vector group”), a group comprising control binary feature vectors (“a control binary feature vector group”), and a control pool comprising all available binary feature vectors that are not comprised in one or more of the exposed binary feature vector group and the control binary feature vector group;
determining, by the control group builder, using the exposed binary feature vector group and using the control binary feature vector group, an actual index vector, the actual index vector comprising a measure of a difference between the binary feature vectors corresponding to the exposed users (“exposed binary feature vectors”) and the binary feature vectors corresponding to the control users (“control binary feature vectors”);
determining, by the control group builder, a dot product between the actual index vector and a binary feature vector comprised in the control pool;
storing, by the control group builder, the dot product in the control group storage;
selecting, by the control group builder, a best control binary feature vector having a maximum value of the dot product;
transferring, by the control group builder, the best control binary feature vector from the control pool to the control group;
storing, by the control group builder, the identifier of the new control user (“control user identifier”) in the control group storage;
determining, by the measurement platform, that a size of the control group comprises at least a required control group size;
receiving, by the measurement platform, an advertisement request copy from the SSP;
sending, by the measurement platform, to the SSP a measurement platform bid that wins the auction, the measurement platform bid comprising an amount of money the measurement platform offers to pay for the opportunity to display on the user device a survey configured to measure on behalf of the client DSP performance of the winning advertisement the measurement platform bid further comprising the survey;
sending, by the measurement platform, the survey to the end user;
receiving, by the measurement platform, a response to the survey from the end user;
determining, by the measurement platform, a metric measuring performance of the winning advertisement;
and
sending, by the measurement platform, the metric to the client DSP.

19 more elements

A method for measuring effectiveness of an advertisement using a measurement platform, the method comprising:

receiving, by the measurement platform, the measurement platform configured to measure performance of an advertisement, the measurement platform comprising a demand-side platform (DSP), the measurement platform further comprising a profile storage configured to store an end user profile, the end user profile comprising a set of features associated with the end user, the measurement platform further comprising a control group storage configured to store an identifier of a control user (“control user identifier”), the measurement platform further comprising a control group builder operably connected to the data platform, the control group builder configured to update the control group with similar statistical properties to an exposed group of end users (“exposed users”) that have seen the winning advertisement, an advertisement display notification notifying the measurement platform that a device under control of the end user (“user device”) has displayed to the end user a winning advertisement from a client DSP, the client DSP comprising a DSP of a client of the measurement platform, the winning advertisement having won an auction run by a supply-side platform (SSP) operably connected to the DSP, the advertisement display notification comprising an identifier of the end user;
storing, by the measurement platform, the identifier of the exposed end user (“exposed user identifier”) in the exposed group;
updating, by the exposed group storage, the exposed group;
retrieving, by the control group builder, a selected portion of the exposed user identifiers;
retrieving, by the control group builder, a selected portion of the control user identifiers;
retrieving, by the control group builder, a selected portion of the end user profiles comprised in the profile storage;
encoding, by the control group builder, using one-hot encoding, an end user profile into a binary feature vector, wherein at least one element of the binary feature vector equals 0 or 1;
sorting, by the control group builder, the binary feature vector into one of a group comprising exposed binary feature vectors (“an exposed binary feature vector group”), a group comprising control binary feature vectors (“a control binary feature vector group”), and a control pool comprising all available binary feature vectors that are not comprised in one or more of the exposed binary feature vector group and the control binary feature vector group;
determining, by the control group builder, using the exposed binary feature vector group and using the control binary feature vector group, an actual index vector, the actual index vector comprising a measure of a difference between the binary feature vectors corresponding to the exposed users (“exposed binary feature vectors”) and the binary feature vectors corresponding to the control users (“control binary feature vectors”), using an equation (1):
actual_index
⁢
_vector
i
=
ln
⁢
(
(
expCount
i
⋆
⁢
numConBFVectors
)
(
conCount
i
⋆
⁢
numExpBFVectors
)
)
,
(
1
)
wherein:
expCounti=a count of 1's comprised in an ith element of the exposed binary feature vector;
conCounti=a count of 1's comprised in respective ith element of the control binary feature vectors;
numConBFVectors=number of binary feature vectors comprised in the control group;
and
numExpBFVectors=number of binary feature vectors comprised in the exposed group;
determining, by the control group builder, a dot product between the actual index vector and a binary feature vector comprised in the control pool;
storing, by the control group builder, the dot product in the control group storage;
selecting, by the control group builder, a best control binary feature vector having a maximum value of the dot product;
transferring, by the control group builder, the best control binary feature vector from the control pool to the control group;
storing, by the control group builder, the identifier of the new control user (“control user identifier”) in the control group storage;
determining, by the measurement platform, that a size of the control group does not comprise at least a required control group size;
and
returning, by the measurement platform, to the step of retrieving all available exposed user identifiers.

47 more elements

A method for measuring effectiveness of an advertisement using a measurement platform, the method comprising:

receiving, by the measurement platform, the measurement platform configured to measure performance of an advertisement, the measurement platform comprising a demand-side platform (DSP), the measurement platform further comprising a profile storage configured to store an end user profile, the end user profile comprising a set of features associated with the end user, the measurement platform further comprising a control group storage configured to store an identifier of a control user (“control user identifier”), the measurement platform further comprising a control group builder operably connected to the data platform, the control group builder configured to update the control group with similar statistical properties to an exposed group of end users (“exposed users”) that have seen the winning advertisement, an advertisement display notification notifying the measurement platform that a device under control of the end user (“user device”) has displayed to the end user a winning advertisement from a client DSP, the client DSP comprising a DSP of a client of the measurement platform, the winning advertisement having won an auction run by a supply-side platform (SSP) operably connected to the DSP, the advertisement display notification comprising an identifier of the end user;
storing, by the measurement platform, the identifier of the exposed end user (“exposed user identifier”) in the exposed group;
updating, by the exposed group storage, the exposed group;
retrieving, by the control group builder, a selected portion of the exposed user identifiers;
retrieving, by the control group builder, a selected portion of the control user identifiers;
retrieving, by the control group builder, a selected portion of the end user profiles comprised in the profile storage;
encoding, by the control group builder, using one-hot-encoding, an end user profile into a binary feature vector, wherein at least one element of the binary feature vector equals 0 or 1;
sorting, by the control group builder, the binary feature vector into one of a group comprising exposed binary feature vectors (“an exposed binary feature vector group”), a group comprising control binary feature vectors (“a control binary feature vector group”), and a control pool comprising all available binary feature vectors that are not comprised in one or more of the exposed binary feature vector group and the control binary feature vector group;
determining, by the control group builder, using the exposed binary feature vector group and using the control binary feature vector group, an actual index vector, the actual index vector comprising a measure of a difference between the binary feature vectors corresponding to the exposed users (“exposed binary feature vectors”) and the binary feature vectors corresponding to the control users (“control binary feature vectors”), using an equation (1):
actual_index
⁢
_vector
i
=
ln
⁢
(
(
expCount
i
⋆
⁢
numConBFVectors
)
(
conCount
i
⋆
⁢
numExpBFVectors
)
)
,
(
1
)
wherein:
expCounti=a count of 1's comprised in an ith element of the exposed binary feature vector;
conCounti=a count of 1's comprised in respective ith element of the control binary feature vectors;
numConBFVectors=number of binary feature vectors comprised in the control group;
and
numExpBFVectors=number of binary feature vectors comprised in the exposed group;
determining, by the control group builder, a dot product between the actual index vector and a binary feature vector comprised in the control pool;
storing, by the control group builder, the dot product in the control group storage;
selecting, by the control group builder, a best control binary feature vector having a maximum value of the dot product;
transferring, by the control group builder, the best control binary feature vector from the control pool to the control group;
storing, by the control group builder, the identifier of the new control user (“control user identifier”) in the control group storage;
determining, by the measurement platform, that a size of the control group comprises at least a required control group size;
receiving, by the measurement platform, an advertisement request copy from the SSP;
sending, by the measurement platform, to the SSP a measurement platform bid that wins the auction, the measurement platform bid comprising an amount of money the measurement platform offers to pay for the opportunity to display on the user device a survey configured to measure on behalf of the client DSP performance of the winning advertisement the measurement platform bid further comprising the survey;
sending, by the measurement platform, the survey to the end user;
receiving, by the measurement platform, a response to the survey from the end user;
determining, by the measurement platform, a metric measuring performance of the winning advertisement;
and
sending, by the measurement platform, the metric to the client DSP.

52 more elements

Dependencies are inferred from the claim text.