Multi-objective off-policy learning for page composition

A non-transitory, computer-readable storage media comprising computer-executable instructions for selecting a plurality of media carousels, wherein the computer-executable instructions, when executed by a computing system, cause the computing system to:

• generate a relevance-ordered list of media carousels comprising a plurality of media carousels comprising a plurality of media types;
• collect historical log information comprising a plurality of displayed pages, wherein each of the displayed pages comprises a corresponding set of media carousels, and wherein each of the corresponding set of media carousels comprises at least one media object;
• assign a reward value to each of the plurality of displayed pages to generate a reward vector, the reward value based on a user interaction associated with each displayed page, wherein the reward vector comprises an indication of whether a user selected a media object from the corresponding set of media carousels of the plurality of displayed pages;
• estimate a logging policy based in part on the historical log information, wherein the logging policy is estimated based in part on a loss function;
• estimate a target policy based in part on the reward vector, wherein the target policy is determined based in part on a reward value;
• train a carousel selection model based on the target policy and the logging policy;
• apply user information, third-party information, and a page template to the carousel selection model to generate a result;
• select, based in part on the result, a plurality of media carousels from the relevance-ordered list comprising a first media type and a second media type, wherein a number of media carousels selected is based on the page template;
• and
• cause a user device to display the plurality of media carousels.

A system comprising:

• a memory storing computer-executable instructions;
• a processor in communication with the memory, wherein the computer-executable instructions when executed by the processor cause the processor to:
• generate a relevance-ordered list of media carousels comprising a plurality of media carousels comprising a plurality of media types;
• collect historical log information comprising a plurality of displayed pages, wherein each of the displayed pages of the plurality of displayed pages comprises a corresponding set of media carousels, and wherein each of the corresponding set of media carousels comprises at least one media object;
• assign a reward value to each of the plurality of displayed pages to generate a reward vector, the reward value based on a user interaction associated with each displayed page, wherein the reward vector comprises an indication of whether a user selected a media object from the corresponding set of media carousels of the plurality of displayed pages;
• estimate a logging policy based in part on the historical log information, wherein the logging policy is estimated based in part on a loss function;
• estimate a target policy based in part on the reward vector, wherein the target policy is determined based in part on a reward value;
• train a carousel selection model based on the target policy and the logging policy;
• apply a user information associated with a user, a third-party information item associated with a third party, and a page template to the carousel selection model to generate a result;
• select, based in part on the result, a plurality of media carousels from the relevance-ordered list comprising a first media type and a second media type, wherein a number of media carousels selected is based on the page template;
• and
• cause a user device to display the plurality of media carousels.