Data efficient imitation of diverse behaviors

A system for reinforcement learning, the system comprising:

• the encoder of a trained variational autoencoder neural network, the encoder comprising a recurrent neural network to encode a probability distribution of the trajectories as an embedding vector defining parameters representing the probability distribution;
• wherein the reinforcement learning system is configured to:
• determine a target embedding vector for a target trajectory by sampling from the probability distribution encoded for the target trajectory by the encoder;
• and
• train a reinforcement learning neural network using reward values conditioned on the target embedding vector.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for training a neural network used to select actions to be performed by an agent interacting with an environment, the operations comprising:

• obtaining data identifying a set of trajectories, each trajectory comprising a set of observations characterizing a set of states of the environment and corresponding actions performed by another agent in response to the states;
• obtaining data identifying an encoder that maps the observations onto embeddings for use in determining a set of imitation trajectories;
• determining, for each trajectory, a corresponding embedding by applying the encoder to the trajectory;
• determining a set of imitation trajectories by applying a policy defined by the neural network to the embedding for each trajectory;
• and
• adjusting parameters of the neural network based on the set of trajectories, the set of imitation trajectories and the embeddings.