Ranking user comments on media using reinforcement learning optimizing for session dwell time

A non-transitory computer readable medium having program instructions embodied thereon, the program instructions being configured, when executed by a computing device, to cause the computing device to perform operations comprising:

• using a scoring model to score each comment, of comments, based on corresponding features of the comment;
• receiving a request to serve a subset of the comments;
• determining a plurality of possible rankings of the comments associated with a plurality of possible permutations;
• responsive to the request, selecting a ranking of the comments that is one permutation from the plurality of possible rankings of the comments, wherein selecting the ranking is in accordance with a probability distribution of the plurality of possible rankings that is based on scores of the comments, wherein the ranking of the comments is associated with increasing a reward at a given time by representing the comments in an order of the ranking, receiving one or more measurable reactions as a scalar reward for the ranking, and updating a ranking mechanism to increase the reward;
• and
• serving one or more comments identified by the selected ranking over a network to a client device,
• wherein at least one of:
• the operations comprise adjusting the scoring model to maximize one or more dwell times of comments ordered according to one or more scores;
• or
• the probability distribution is defined from probabilities of permutations of the plurality of possible rankings, such that a given ranking prioritizing comments predicted to have greater dwell time has a higher probability than a given ranking prioritizing comments predicted to have a lower dwell time.

A system comprising at least one server computer, the at least one server computer having:

• logic for using a scoring model to score each comment, of comments, based on corresponding features of the comment, wherein an objective associated with the scoring model is changed without re-training the scoring model;
• logic for receiving a request to serve a subset of the comments;
• logic for determining a plurality of possible rankings of the comments associated with a plurality of possible permutations;
• logic for, responsive to the request, selecting a ranking of the comments that is one permutation from the plurality of possible rankings of the comments, wherein selecting the ranking is in accordance with a probability distribution of the plurality of possible rankings that is based on scores of the comments, wherein the ranking of the comments is associated with increasing a reward at a given time by representing the comments in an order of the ranking, receiving one or more measurable reactions as a scalar reward for the ranking, and updating a ranking mechanism to increase the reward;
• and
• logic for serving one or more comments identified by the selected ranking over a network to a client device.