Controlling a swarm of agents

A computer program product comprising one or more non-transitory machine readable mediums encoded with instructions which, when executed by one or more processors, cause a process to be carried out for controlling components associated with a swarm of agents by controlling the swarm of agents to perform actions, the process comprising:

• providing data for performing reinforcement learning in each agent in the swarm:
• each of the agents in the swarm using the provided data to execute at least one learning episode to train a respective state-action selector in each of the agents in the swarm, wherein the state-action selector is configured to select an action based on joint states of the agents in the swarm;
• and
• each of the agents in the swarm using its respective trained state-action selector to select and perform an action and sending a control signal corresponding to the selected action to a said component associated with the agent;
• wherein the reinforcement learning comprises a Markov decision process (MDP), and the provided data includes data for performing the MDP and includes
• the state-action selector,
• a policy for determining how to use the state-action selector to select the action based on the joint state of the agents, the policy comprising an ε-greedy policy,
• a probabilistic behaviour model configured to predict a next state of an environment based on a current state of the environment, wherein the probabilistic behaviour model comprises an artificial neural network configured to predict next actions of non-controllable items/features in the environment given current joint states of the agents, and
• an environment model configured to generate a set of next joint states of the agents and rewards, based on the current state of the environment, the current joint states of the agents, and current actions of the agents in the swarm;
• and
• wherein the training of the respective state-action selector in each of the agents in the swarm includes
• using the probabilistic behaviour model to predict the next state of the environment,
• each of the agents in the swarm using its respective state-action selector to select a fictitious action it would perform itself in the current joint states and to select a fictitious action other agents in the swarm would perform in the current joint states,
• using the environment model to generate a next reward and a next joint states transition of the agents based on the predicted next state of the environment,
• storing an MDP sequence comprising the selected fictitious actions, the next reward, and the next joint state transition, and
• using the selected fictitious actions to update values in the state-action selector of each of the agents in the swarm.

The computer program product according to claim 13, the process comprising:

• each of the agents in the swarm using its respective policy and its respective state-action selector to select an action to perform in the environment;
• transitioning the environment to the next state, providing a corresponding reward to each of the agents in the swarm, and storing an MDP sequence to update values in the state-action selectors;
• and
• each of the agents in the swarm using the stored MDP sequence to update the values in its respective state-action selector.

A computer program product comprising one or more non-transitory machine readable mediums encoded with instructions which, when executed by one or more processors, cause a process to be carried out for controlling an agent to perform actions, the agent included in a swarm of agents and includes a state-action selector that is trained to select an action based on joint states of the agents in the swarm, the process comprising:

• selecting and performing, by the agent included in the swarm of agents and using its trained state-action selector, an action;
• and
• sending, by the agent included in the swarm of agents, a control signal corresponding to the selected action to a component associated with the agent included in the swarm of agents;
• wherein the state-action selector is trained using reinforcement learning, and the reinforcement learning comprises a Markov decision process (MDP).