Real-time real-world reinforcement learning systems and methods

A system, comprising:

• one or more communication interfaces configured to communicate over one or more networks;
• one or more processors coupled to the one or more communication interfaces;
• memory storing a set of instructions that cause the one or more processors to establish, via the one or more communication interfaces, communications with an external real-time system comprising a plurality of robots and that cause the one or more processors to execute processes that include:
• a first communicator configured to receive first state data that indicates a state of a first robot of the plurality of robots, and cause the first robot to be controlled to take an action by transmitting a first actuation command to the first robot via the one or more communication interfaces;
• a second communicator configured to receive second state data that indicates a state of a second robot of the plurality of robots, and to cause the second robot to be controlled to take an action by transmitting a second actuation command to the second robot via the one or more communication interfaces;
• a task manager configured to:
• generate a joint state vector based on the first state data and the second state data;
• and
• parse a joint action vector into the first actuation command and the second actuation command;
• and
• a reinforcement learning agent configured to:
• transition from a suspended state, in which the reinforcement learning agent operates for a predefined period of time and does not obtain or process another joint state vector, to an active state;
• generate, in the active state, the joint action vector based on the joint state vector;
• and
• transition, in response to generating the joint action vector, back to the suspended state.

At least one non-transitory computer-readable media storing a set of instructions that, as a result of execution by one or more processors of a computer system, cause the one or more processors to implement a plurality of processes that include:

• a first communicator process that receives first state data that indicates a state of a first robot operating in a physical environment, and causes the first robot to be controlled to take an action by transmitting a first actuation command to the first robot;
• a second communicator process that receives second state data that indicates a state of a second robot operating in the physical environment, and causes the second robot to be controlled to take an action by transmitting a second actuation command to the second robot;
• a task manager process that:
• generates a joint state vector based on the first state data and the second state data;
• and
• parses a joint action vector into the first actuation command and the second actuation command;
• and
• a reinforcement learning agent process that:
• transitions from a suspended state, in which the reinforcement learning agent operates for a predefined period of time and does not obtain or process another joint state vector, to an active state;
• generates, in the active state, the joint action vector based on the joint state vector;
• and
• transition, in response to generating the joint action vector, back to the suspended state.