US 10,706,454 B2Granted

Method, medium, and system for training and utilizing item-level importance sampling models

AssigneeAdobe Inc.

InventorsShuai Li, Zheng Wen, Yasin Abbasi Yadkori, Vishwa Vinay, Branislav Kveton

Life of this patent

Estimating the term…

Loading patent…

Abstract

The present disclosure is directed toward systems, methods, and computer readable media for training and utilizing an item-level importance sampling model to evaluate and execute digital content selection policies. For example, systems described herein include training and utilizing an item-level importance sampling model that accurately and efficiently predicts a performance value that indicates a probability that a target user will interact with ranked lists of digital content items provided in accordance with a target digital content selection policy. Specifically, systems described herein can perform an offline evaluation of a target policy in light of historical user interactions corresponding to a training digital content selection policy to determine item-level importance weights that account for differences in digital content item distributions between the training policy and the target policy. In addition, the systems described herein can apply the item-level importance weights to training data to train item-level importance sampling model.

Claims20

Dependent claims

In a digital medium environment for selecting and providing digital item lists to client devices in accordance with content selection policies, a computer-implemented method of training a policy model for offline evaluation and execution of a content selection policy, the method comprising:

identifying digital interactions by users with respect to digital content items, the digital content items selected and presented as part of training item lists to computing devices of the users in accordance with a training digital content selection policy;
training an item-level importance sampling model to predict a performance value indicating a probability that a user will interact with one or more items from a target item list selected in accordance with a target digital content selection policy by:
for a first digital content item of the digital content items, determining an item-level importance weight based on the training digital content selection policy and the target digital content selection policy;
and
applying the item-level importance weight to a first interaction from the identified digital interactions by a first computing device of a first user with the first digital content item from a first item list to train the item-level importance sampling model;
training the item-level importance sampling model to predict a second performance value indicating a second probability that a user will interact with one or more items from a second target item list selected in accordance with a second target digital content selection policy;
and
comparing the performance value associated with the target digital content selection policy with the second performance value associated with the second target digital content selection policy.

5 more elements

A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computer system to:

identify digital interactions by users with respect to digital content items, the digital content items selected and presented as part of training item lists to computing devices of the users in accordance with a training digital content selection policy;
and
train an item-level importance sampling model to predict a performance value indicating a probability that a user will interact with one or more items from a target item list selected in accordance with a target digital content selection policy by:
for a first digital content item of the digital content items, determining an item-level importance weight based on the training digital content selection policy and the target digital content selection policy;
and
applying the item-level importance weight to a first interaction from the identified digital interactions by a first computing device of a first user with the first digital content item from a first item list to train the item-level importance sampling model;
train the item-level importance sampling model to predict a second performance value indicating a second probability that a user will interact with one or more items from a second target item list selected in accordance with a second target digital content selection policy;
and
compare the performance value associated with the target digital content selection policy with the second performance value associated with the second target digital content selection policy.

6 more elements

A system comprising:

at least one processor;
a non-transitory computer readable medium comprising instructions that, when executed by the at least one processor, cause the system to:
identify digital interactions by users with respect to digital content items, the digital content items selected and presented as part of training item lists to computing devices of the users in accordance with a training digital content selection policy;
train an item-level importance sampling model to generate a performance value that indicates a probability that a user will interact with one or more items from a target item list selected in accordance with a target digital content selection policy by:
for a first digital content item of the digital content items, determining an item-level importance weight based on the training digital content selection policy and the target digital content selection policy;
and
applying the item-level importance weight to a first interaction from the digital interactions by a first computing device of a first user with the first digital content item from a first item list train the item-level importance sampling model;
train the item-level importance sampling model to predict a second performance value indicating a second probability that a user will interact with one or more items from a second target item list selected in accordance with a second target digital content selection policy;
and
compare the performance value associated with the target digital content selection policy with the second performance value associated with the second target digital content selection policy.

7 more elements

Dependencies are inferred from the claim text.