Controlling robots using latent action vector conditioned controller neural networks

A system comprising:

• one or more computers;
• and
• one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations for controlling an agent interacting with an environment to perform a task, the operations comprising, at each of a plurality of time steps:
• receiving an observation comprising data characterizing a state of the environment at the time step, wherein the data characterizing the state of the environment comprises sensor data generated from sensor readings of sensors of the agent at the time step;
• processing the observation using a task policy neural network for the task to generate a task output that defines a latent action vector from a latent action space;
• processing a low-level input comprising (i) the sensor data and (ii) the latent action vector defined by the task output using a low-level controller neural network to generate a policy output that defines a control input for controlling the agent in response to the observation, wherein the low-level controller neural network is configured to:
• process the sensor data through a first neural network branch comprising a plurality of first neural network layers to generate a first branch output;
• process a second branch input comprising the first branch output and the latent action vector defined by the task output through a second neural network branch comprising a plurality of second neural network layers to generate a second branch output;
• and
• generate the policy output from the first branch output and the second branch output;
• and
• controlling the agent using the control input defined by the policy output.