Hierarchical latent mixture policies for agent control

A system comprising:

• one or more computers;
• and
• one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations for controlling an agent interacting with an environment to perform a task, the operations comprising, at each of a plurality of times steps:
• receiving an observation characterizing a state of the environment at the time step;
• processing a high-level input derived from the observation using a high-level controller neural network to generate a high-level output comprising a respective score for each skill in a set of skills,
• selecting, using the high-level output, a skill from the set of skills;
• process a mid-level input derived from the observation using a mid-level controller neural network and while the mid-level controller neural network is conditioned on the selected skill to generate a latent action vector from a latent action space, and
• processing a low-level input derived from the observation and the latent action vector using a low-level controller neural network to generate a policy output that defines an action to be performed by the agent in response to the observation.

One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for controlling an agent interacting with an environment to perform a task, the operations comprising, at each of a plurality of time steps:

• receiving an observation characterizing a state of the environment at the time step,
• processing a high-level input derived from the observation using a high-level controller neural network to generate a high-level output comprising respective score for each skill in a set of skills,
• selecting, using the high-level output, a skill from the set of skills;
• processing a mid-level input derived from the observation using a mid-level controller neural network and while the mid-level controller neural network is conditioned on the selected skill to generate a latent action vector from a latent action space, and
• processing a low-level input derived from the observation and the latent action vector using a low-level controller neural network to generate a policy output that defines an action to be performed by the agent in response to the observation.