System and methods for training robot policies in the real world

A system comprising:

• one or more processors;
• and
• memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations that include:
• determining a pose of a mobile robot within a real world training workspace;
• selecting, from a plurality of disparate policy networks each being fora corresponding component of locomotion, a corresponding policy network, wherein selecting the corresponding policy network is based on comparing the pose to a position of a center of the real world training workspace to determine the corresponding policy network with the corresponding component of locomotion that will move the mobile robot towards the center of the real world training workspace;
• and
• for each of a plurality of iterations, and until one or more conditions are satisfied:
• determining current state data of the mobile robot,
• using the selected policy network and the corresponding current state data to determine one or more corresponding actions to move the mobile robot towards the center of the real world training workspace,
• storing a corresponding training instance in association with the selected corresponding policy network, the corresponding training instance including at least the corresponding current state data and the one or more corresponding actions, and
• implementing the one or more corresponding actions at the mobile robot;
• updating one or more portions of the selected policy network using the training instances stored in association with the selected policy network in the iterations;
• after cessation of the plurality of iterations:
• determining an additional pose of the mobile robot within the real world training workspace, wherein the additional pose is distinct from the pose of the mobile robot, and wherein the additional pose of the mobile robot corresponds with an additional component of locomotion distinct from the component of locomotion of the pose of the mobile robot;
• selecting, from the plurality of disparate policy networks, an additional policy network corresponding to the additional component of locomotion,
• wherein selecting the additional policy network is based on comparing the additional pose to the center of the real world training workspace with the corresponding additional component of locomotion that will move the mobile robot towards the center of the real world training workspace,
• wherein the additional policy network is distinct from the policy network, and
• wherein the additional component of locomotion corresponding to the additional policy network is distinct from the component of locomotion corresponding to the policy network;
• for each of a plurality of additional iterations, and until the one or more conditions are satisfied:
• determining additional current state data of the mobile robot,
• using the selected additional policy network and the corresponding additional current state data to generate one or more corresponding additional actions to move the mobile robot towards the center of the real world training workspace,
• storing an additional corresponding training instance in association with the selected corresponding additional policy network, the corresponding additional training instance including at least the corresponding additional current state data and the one or more corresponding additional actions, and
• implementing the one or more corresponding additional actions at the mobile robot;
• and
• updating one or more portions of the selected additional policy network using the additional training instances stored in association with the selected additional policy network in the additional iterations.