Generating digital recommendations utilizing collaborative filtering, reinforcement learning, and inclusive sets of negative feedback

A computer-implemented method comprising:

• monitoring user interactions of a client device with one or more digital content items from a plurality of digital content items during an interaction session;
• determining, utilizing a set of item embeddings that encode interactions with the plurality of digital content items across client devices, a negative interaction map and a positive interaction map from the user interactions of the client device during the interaction session by:
• determining the positive interaction map or the negative interaction map using, from the set of item embeddings, item embeddings that correspond to the user interactions of the client device during the interaction session;
• and
• determining at least one of the positive interaction map or the negative interaction map using, from the set of item embeddings, additional item embeddings selected based on distances within an item embedding space between the additional item embeddings and the item embeddings that correspond to the user interactions, the additional item embeddings corresponding to digital content items with which the client device did not interact during the interaction session;
• generating, utilizing a first convolutional gated recurrent unit neural network layer of a reinforcement learning model, a negative state for the client device based on the negative interaction map;
• generating, utilizing a second convolutional gated recurrent unit neural network layer of the reinforcement learning model, a positive state for the client device based on the positive interaction map;
• determining, utilizing the reinforcement learning model, one or more additional digital content items from the plurality of digital content items based on the set of item embeddings, the negative state, and the positive state;
• providing the one or more additional digital content items for display via the client device;
• determining, using one or more rectified linear interaction neural network layers of the reinforcement learning model, a value function indicating a measure of quality of the one or more additional digital content items provided for display;
• and
• modifying parameters of the reinforcement learning model using the value function.

A system comprising:

• one or more memory devices;
• and
• one or more server devices configured to cause the system to:
• generate interaction maps from user interactions of a client device with one or more digital content items from a plurality of digital content items during an interaction session by:
• determining a positive interaction map or a negative interaction map using item embeddings that correspond to the user interactions of the client device during the interaction session;
• and
• determining at least one of the positive interaction map or the negative interaction map using additional item embeddings selected based on distances within an item embedding space between the additional item embeddings and the item embeddings that correspond to the user interactions, the additional item embeddings corresponding to digital content items with which the client device did not interact during the interaction session;
• determine, utilizing an actor model of a reinforcement learning model and based on the interaction maps, one or more additional digital content items from the plurality of digital content items to provide for display via the client device by:
• generating, utilizing a first convolutional gated recurrent unit neural network layer of the actor model, a negative state for the client device based on the interaction maps;
• and
• generating, utilizing a second convolutional gated recurrent unit neural network layer of the actor model, a positive state for the client device based on the interaction maps;
• determining to provide the one or more additional digital content items for display via the client device based on the positive state and the negative state;
• and
• generate, utilizing a critic model of the reinforcement learning model, a value function to modify parameters of the actor model by:
• determining a first state-action vector utilizing the positive state and a second state-action vector utilizing the negative state;
• and
• generating, utilizing one or more rectified linear interaction neural network layers of the critic model, the value function based on the first state-action vector and the second state-action vector.