Imagination-based agent neural networks

A method of training a neural network system, wherein the neural network system is used to select actions to be performed by an agent interacting with an environment to perform a task in an attempt to achieve a specified result, the system comprising:

• at least one imagination core comprising a model of said environment and having an input to receive at least a current observation characterizing a current state of the environment, wherein the imagination core is configured to output trajectory data in response to the current observation, the trajectory data defining a trajectory comprising a sequence of future features of the environment imagined by the imagination core;
• at least one rollout encoder to encode the sequence of features from the imagination core to provide a rollout embedding for the trajectory;
• and
• a reinforcement learning output stage to receive data derived from the rollout embedding and to output action policy data for defining an action policy identifying an action based on said current observation, and the method comprising:
• pre-training one or more models of said environment for said imagination cores and then training another part of the neural network system using reinforcement learning.

One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to implement a neural network system for model-based reinforcement learning, wherein the neural network system is used to select actions to be performed by an agent interacting with an environment to perform a task in an attempt to achieve a specified result, the system comprising:

• at least one imagination core comprising a model of said environment and having an input to receive at least a current observation characterizing a current state of the environment, wherein the imagination core is configured to output trajectory data in response to the current observation, the trajectory data defining a trajectory comprising a sequence of future features of the environment imagined by the imagination core;
• at least one rollout encoder to encode the sequence of features from the imagination core to provide a rollout embedding for the trajectory;
• and
• a reinforcement learning output stage to receive data derived from the rollout embedding and to output action policy data for defining an action policy identifying an action based on said current observation.