Determining performance metrics for delivery of electronic media content items by online publishers scaled using a baseline conversion rate

A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

• store, by an online system, information describing deliveries of content items and user actions associated with the delivered content items, each delivery performed by one of a plurality of online publishers to one of a plurality of users of the online system;
• detect a user action associated with a content item performed by a target user of the plurality of users;
• retrieve, from the stored information, information describing a set of the plurality of online publishers that delivered the content item to the target user;
• retrieve a set of attributes maintained in the online system for the target user;
• extract a set of feature vectors from the stored information describing deliveries of content items and user actions associated with the delivered content items, each feature vector identifying online publishers that delivered one or more content items associated with the user action;
• generate training sets from the feature vectors of the set, each training set including a set of features and a label representing whether delivery of one or more content items resulted in the user action;
• train a machine learning model that receives a feature vector and generates a likelihood that the user action would have been performed by a user without an online publisher's delivery of the one or more content items to the user by applying the machine learning model to the generated training sets and re-training the machine learning model by application to the generated training sets until occurrence of a stopping condition;
• for each online publisher of the set of the plurality of online publishers,
• extract a feature vector from the retrieved information describing the set of the plurality of online publishers that delivered the content item to the target user and describing the set of attributes for the target user,
• provide the extracted feature vector to the machine learning model that generates a likelihood that the user action would have been performed by the target user without the online publisher's delivery of the content item to the target user,
• generate, by the machine learning model based on the extracted feature vector, the likelihood for the online publisher, and
• determine an estimated increase in the likelihood that the user action was performed by the target user due to the online publisher's delivery of the content item to the target user based on the generated likelihood that the user action would have been performed by the target user without the online publisher's delivery of the content item to the target user;
• estimate, using the set of attributes for the target user, a baseline value indicating an additional likelihood that the user action would have been performed by the target user without delivery of the content item to the target user by any of the set of the plurality of online publishers;
• determine a normalized performance metric for each online publisher relative to the baseline value, wherein the normalized performance metrics are determined by comparing:
• the estimated increase in the likelihood that the user action was performed by the target user due to the online publisher's delivery of the content item to the target user normalized relative to the estimated increases in the likelihoods for the set of the plurality of online publishers;
• and
• a sum of normalized estimated increases in the likelihoods for the set of the plurality of online publishers and the baseline value;
• transmit the performance metrics for the set of the plurality of online publishers for the user action.