Training data generation by bucketing users based on output of a contextual bandit model

A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to perform steps comprising:

• accessing a contextual bandit model, wherein the contextual bandit model is a machine-learning model trained to compute a likelihood of presenting each content item in a set of content items to a respective user of a plurality of users of an online system;
• applying the contextual bandit model to user data for each of the plurality of users to generate the likelihood of presenting each content item to the respective user;
• grouping, based at least in part on the likelihood of presenting each content item in the set of content items to the respective user, each of the plurality of users into a corresponding bucket of a plurality of buckets, the corresponding bucket being associated with target rate of presenting each content item in the set of content items to the plurality of users;
• selecting, based at least in part on the target rate and a first set of features of the plurality of users, a corresponding content item from the set of content items for presentation to a first set of users of the plurality of users;
• upon presentation of the corresponding content item to the first set of users, obtaining information about a first rate of presenting the corresponding content item to the plurality of users;
• adjusting, based at least in part on the first rate, a value of the target rate;
• generating, based at least in part on the adjusted value of the target rate of presenting each content item in the set of content items to the plurality of users, training data for a ranking model;
• training, using the generated training data, a set of parameters of the ranking model;
• accessing the ranking model, wherein the ranking model is a machine-learning model trained to generate a ranking score for each content item in the set of content items;
• applying the ranking model to user data for a user of the online system and contextual data associated with a current session of the user to generate the ranking score for each content item in the set of content items;
• selecting, based on the ranking score for each content item in the set of content items, one or more content items from the set of content items;
• and
• causing a device associated with the user to display a user interface with the one or more content items for recommendation to the user, wherein an actual rate of presenting each of the one or more content items is within a threshold difference from the target rate of presenting each of the one or more content items due to training of the set of parameters of the ranking model using the generated training data.

A computer system comprising:

• a processor;
• and
• a non-transitory computer-readable storage medium having instructions that, when executed by the processor, cause the computer system to perform steps comprising:
• accessing a contextual bandit model, wherein the contextual bandit model is a machine-learning model trained to compute a likelihood of presenting each content item in a set of content items to a respective user of a plurality of users of an online system;
• applying the contextual bandit model to user data for each of the plurality of users to generate the likelihood of presenting each content item to the respective user;
• grouping, based at least in part on the likelihood of presenting each content item in the set of content items to the respective user, each of the plurality of users into a corresponding bucket of a plurality of buckets, the corresponding bucket being associated with target rate of presenting each content item in the set of content items to the plurality of users;
• selecting, based at least in part on the target rate and a first set of features of the plurality of users, a corresponding content item from the set of content items for presentation to a first set of users of the plurality of users;
• upon presentation of the corresponding content item to the first set of users, obtaining information about a first rate of presenting the corresponding content item to the plurality of users;
• adjusting, based at least in part on the first rate, a value of the target rate;
• generating, based at least in part on the adjusted value of the target rate of presenting each content item in the set of content items to the plurality of users, training data for a ranking model;
• training, using the generated training data, a set of parameters of the ranking model;
• accessing the ranking model, wherein the ranking model is a machine-learning model trained to generate a ranking score for each content item in the set of content items;
• applying the ranking model to user data for a user of the online system and contextual data associated with a current session of the user to generate the ranking score for each content item in the set of content items;
• selecting, based on the ranking score for each content item in the set of content items, one or more content items from the set of content items;
• and
• causing a device associated with the user to display a user interface with the one or more content items for recommendation to the user, wherein an actual rate of presenting each of the one or more content items is within a threshold difference from the target rate of presenting each of the one or more content items due to training of the set of parameters of the ranking model using the generated training data.