Determining action selection policies of an execution device

A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:

• computing a hybrid sampling policy at a state of an execution device comprising one or more processors based on a sampling policy and an exploration policy, wherein the state corresponds to a plurality of possible actions that lead to respective next states if performed by the execution device when the execution device is in the state, wherein the sampling policy specifies a respective sampling probability of sampling each of the plurality of possible actions in the state;
• wherein the exploration policy specifies a respective exploration probability corresponding to each of the plurality of possible actions in the state, wherein an exploration probability of an action in the plurality of possible actions is negatively correlated with a number of times that the action in the state has been sampled, such that the exploration probability of the action decreases as the number of times that the action in the state has been sampled increases;
• sampling a first action among the plurality of possible actions in the state according to a sampling probability of the first action specified in the hybrid sampling policy;
• updating an action selection policy of the execution device in the state by performing Monte Carlo counterfactual regret minimization (MCCFR) based on the first action, wherein the action selection policy specifies a respective probability of selecting a respective action among the plurality of possible actions in the state for completing a task in an environment that includes the execution device and one or more other devices;
• determining that a convergence condition is met;
• and
• in response to determining that the convergence condition is met, controlling operations of the execution device based on the action selection policy.

A computer-implemented system, comprising:

• one or more computers;
• and
• one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising:
• computing a hybrid sampling policy at a state of an execution device comprising one or more processors based on a sampling policy and an exploration policy, wherein the state corresponds to a plurality of possible actions that lead to respective next states if performed by the execution device when the execution device is in the state, wherein the sampling policy specifies a respective sampling probability of sampling each of the plurality of possible actions in the state;
• wherein the exploration policy specifies a respective exploration probability corresponding to each of the plurality of possible actions in the state, wherein an exploration probability of an action in the plurality of possible actions is negatively correlated with a number of times that the action in the state has been sampled, such that the exploration probability of the action decreases as the number of times that the action in the state has been sampled increases;
• sampling a first action among the plurality of possible actions in the state according to a sampling probability of the first action specified in the hybrid sampling policy;
• updating an action selection policy of the execution device in the state by performing Monte Carlo counterfactual regret minimization (MCCFR) based on the first action, wherein the action selection policy specifies a respective probability of selecting a respective action among the plurality of possible actions in the state for completing a task in an environment that includes the execution device and one or more other devices;
• determining that a convergence condition is met;
• and
• in response to determining that the convergence condition is met, controlling operations of the execution device based on the action selection policy.