Selecting content items using reinforcement learning

One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to implement:

• a long-term engagement machine learning model, the long-term engagement machine learning model having been trained through reinforcement learning to:
• receive a plurality of inputs, each input including data characterizing a respective context in which a respective content item may be presented in a presentation environment to a respective user, and
• process each of the plurality of inputs to generate a respective engagement score for each input that represents a predicted, time-adjusted total number of selections by the respective user of future content items presented to the respective user in the presentation environment if the respective content item is presented in the respective context;
• and
• a subsystem configured to perform operations comprising:
• receiving first data characterizing a first context in which a first content item may be presented to a first user in the presentation environment;
• providing the first data as input to the long-term engagement machine learning model to obtain a first engagement score that represents a predicted, time-adjusted total number of selections by the first user of future content items presented to the first user in the presentation environment if the first content item is presented in the first context;
• and
• determining, from the first engagement score, whether or not to present the first content item to the first user in the first context.

A method performed by one or more computers, the method comprising:

• receiving first data characterizing a first context in which a first content item may be presented to a first user in a presentation environment;
• providing the first data as input to a long-term engagement machine learning model to obtain a first engagement score that represents a predicted, time-adjusted total number of selections by the first user of future content items presented to the first user in the presentation environment if the first content item is presented in the first context, the long-term engagement machine learning model having been trained through reinforcement learning to:
• receive a plurality of inputs, each input including data characterizing a respective context in which a respective content item may be presented in the presentation environment to a respective user, and
• process each of the plurality of inputs to generate a respective engagement score for each input that represents a predicted, time-adjusted total number of selections by the respective user of future content items presented to the respective user in the presentation environment if the respective content item is presented in the respective context;
• and
• determining, from the first engagement score, whether or not to present the first content item to the first user in the first context.