Rating tasks and policies using conditional probability distributions derived from equilibrium-based solutions of games

A system comprising:

• one or more computers;
• and
• one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
• determining, for each action selection policy in a pool of action selection policies, a respective performance measure of the action selection policy on each task in a pool of tasks, wherein each action selection policy defines a policy for selecting actions to be performed by an agent in an environment;
• processing the performance measures of the action selection policies on the tasks to generate data defining a joint probability distribution over a set of action selection policy — task pairs,
• wherein each action selection policy — task pair comprises a respective action selection policy from the pool of action selection policies and a respective task from the pool of tasks;
• and
• processing the joint probability distribution over the set of action selection policy — task pairs to generate a respective rating for each action selection policy in the pool of action selection policies, wherein the respective rating for each action selection policy characterizes a utility of the action selection policy in performing tasks from the pool of tasks.

One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• determining, for each action selection policy in a pool of action selection policies, a respective performance measure of the action selection policy on each task in a pool of tasks, wherein each action selection policy defines a policy for selecting actions to be performed by an agent in an environment;
• processing the performance measures of the action selection policies on the tasks to generate data defining a joint probability distribution over a set of action selection policy — task pairs,
• wherein each action selection policy — task pair comprises a respective action selection policy from the pool of action selection policies and a respective task from the pool of tasks;
• and
• processing the joint probability distribution over the set of action selection policy — task pairs to generate a respective rating for each action selection policy in the pool of action selection policies, wherein the respective rating for each action selection policy characterizes a utility of the action selection policy in performing tasks from the pool of tasks.