Training a high-level controller to generate natural language commands for controlling an agent
AssigneeDeepMind Technologies Limited
InventorsArun Ahuja, Robert David Fergus, Ishita Dasgupta, Kavya Venkata Kota Sai Kopparapu
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a high-level controller neural network for controlling an agent. In particular, the high-level controller neural network generates natural language commands that can be provided as input to a low-level controller neural network, which generates control outputs that can be used to control the agent.
Claims20
A method performed by one or more computers and for training a high-level controller neural network that is configured to receive an input comprising an observation characterizing a state of an environment being interacted with by an agent and to generate an output defining a natural language command for a low-level controller neural network that generates control outputs for controlling the agent, the method comprising:
- obtaining a training data set, the training data set comprising:
- a plurality of demonstration trajectories, each demonstration trajectory comprising, for each of a plurality of time steps, a respective observation characterizing a state of a demonstration environment being interacted with by a demonstration agent at the time step and a respective natural language command provided to the demonstration agent at the time step;
- and
- training the high-level controller neural network on the demonstration trajectories in the training data set through supervised learning.
A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for training a high-level controller neural network that is configured to receive an input comprising an observation characterizing a state of an environment being interacted with by an agent and to generate an output defining a natural language command for a low-level controller neural network that generates control outputs for controlling the agent, the operations comprising:
- obtaining a training data set, the training data set comprising:
- a plurality of demonstration trajectories, each demonstration trajectory comprising, for each of a plurality of time steps, a respective observation characterizing a state of a demonstration environment being interacted with by a demonstration agent at the time step and a respective natural language command provided to the demonstration agent at the time step;
- and
- training the high-level controller neural network on the demonstration trajectories in the training data set through supervised learning.
One or more non-transitory computer-readable media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training a high-level controller neural network that is configured to receive an input comprising an observation characterizing a state of an environment being interacted with by an agent and to generate an output defining a natural language command for a low-level controller neural network that generates control outputs for controlling the agent, the operations comprising:
- obtaining a training data set, the training data set comprising:
- a plurality of demonstration trajectories, each demonstration trajectory comprising, for each of a plurality of time steps, a respective observation characterizing a state of a demonstration environment being interacted with by a demonstration agent at the time step and a respective natural language command provided to the demonstration agent at the time step;
- and
- training the high-level controller neural network on the demonstration trajectories in the training data set through supervised learning.
Dependencies are inferred from the claim text.