Selecting digital advertising recommendation policies in light of risk and expected return

In a digital medium environment for identifying and deploying policies, where policies can be altered, removed, or replaced on demand, a method of using risk-sensitive, lifetime value optimization based on a conditional value at risk measure to improve accuracy, efficiency, and stability in selecting and executing a policy comprising:

• identifying, by one or more processors, a risk-tolerance value indicating a measure of permissible variance for policies and a confidence level, the risk-tolerance value and the confidence level corresponding to a conditional value at risk indicating a threshold measure of a mean of an alpha-tail distribution;
• identifying a set of sample data comprising prior interactions by user client devices;
• receiving a set of policies, each policy being defined by one or more policy parameters;
• and
• determining, by the one or more processors, an optimized policy that is subject to the risk-tolerance value within the confidence level using a trajectory-based policy gradient algorithm by converging a policy parameter, a risk parameter indicating policy conditional value at risk, and a constraint parameter by sampling the sample data to identify the optimized policy that satisfies the threshold measure of the mean of the alpha-tail distribution;
• and
• in response to determining the optimized policy, executing the optimized policy subject to the risk-tolerance value and the confidence level corresponding to the conditional value at risk such that digital content is provided to client devices in accordance with the optimized policy.

A system using risk-sensitive, lifetime value optimization based on a conditional value at risk measure to improve accuracy, efficiency, and stability in selecting ad recommendation policies in connection with identifying and deploying digital advertising campaigns, where campaigns can be altered, removed, or replaced on demand, comprising:

• at least one processor;
• at least one non-transitory computer readable storage medium storing instructions thereon, that, when executed by the at least one processor, cause the system to:
• receive a risk-tolerance value and a confidence level for the risk-tolerance value, the risk-tolerance value and confidence level corresponding to a conditional value at risk;
• identify a set of user data indicating prior interactions by user client devices in relation to digital advertisements of one or more digital content campaigns;
• determine an optimized ad recommendation policy that is subject to the risk-tolerance value within the confidence level by converging a policy parameter, a risk parameter indicating policy conditional value at risk, and a constraint parameter to identify the optimized ad recommendation policy such that the optimized ad recommendation policy satisfies the conditional value at risk by repeatedly, for each of the policy parameter, the risk parameter, and the constraint parameter:
• estimating a gradient for a given parameter of the policy parameter, the risk parameter, or the constraint parameter based on sampling of the set of user data with respect to an initial ad recommendation policy;
• and
• using the estimated gradient for the given parameter to update the given parameter;
• and
• in response to determining the optimized ad recommendation policy, execute the digital content campaign subject to the risk-tolerance value and the confidence level corresponding to the conditional value at risk by providing digital advertisements to client devices in accordance with the optimized ad recommendation policy.