Data-efficient hierarchical reinforcement learning

A robot comprising:

• one or more vision components;
• one or more actuators;
• memory storing instructions;
• one or more processors operable to execute the instructions to:
• at a first control step:
• identify a first current state observation;
• determine, using a higher-level policy model, a higher-level action for transitioning from the first current state to a goal state;
• generate a first lower-level action based on processing, using a lower-level policy model, the first current state observation and the higher-level action;
• and
• apply the first lower-level action to one or more of the actuators to cause the robot to transition to an updated state;
• at a second control step:
• identify a second current state observation;
• generate a second lower-level action based on processing, using the lower-level policy model, the second current state observation and the higher-level action that was determined in the first control step;
• and
• apply the second lower-level action to one or more of the actuators to cause the robot to transition to a further updated state;
• and
• at a subsequent control step that is subsequent to the second control step:
• identify a subsequent current state observation;
• determine, using the higher-level policy model, a subsequent higher-level action for transitioning from the subsequent current state to the goal state;
• generate a subsequent lower-level action based on processing, using the lower-level policy model, the subsequent current state observation and the subsequent higher-level action;
• and
• apply the subsequent lower-level action to one or more of the actuators to cause the robot to transition to a subsequent updated state.