Interactive autonomous vehicle agent

A system comprising:

• one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to implement:
• a vehicle behavior model neural network configured to generate a predicted environment observation representing a predicted environment of an autonomous vehicle in a particular driving context after a candidate action is taken by the autonomous vehicle in an initial environment;
• a quality model engine configured to generate an immediate quality value by providing the predicted environment of the autonomous vehicle and the candidate action as input to a context-specific quality model that generates immediate quality values that are specific to the particular driving context;
• and
• a reinforcement learning system for the autonomous vehicle that is configured to receive an experience tuple comprising the initial environment observation, the candidate action, and the immediate quality value and to generate a cumulative action value for the candidate action when performed by the autonomous vehicle in the initial environment.

A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• receiving a request to generate an experience tuple for a vehicle in a particular driving context, the particular driving context being one of a plurality of driving contexts;
• receiving an initial environment observation representing an initial environment of the autonomous vehicle and a candidate action representing an action to be taken by the autonomous vehicle in the initial environment;
• generating a predicted environment observation representing a predicted environment of the autonomous vehicle after the candidate action is taken by the autonomous vehicle in the initial environment, including providing the initial environment observation and the candidate action as input to a vehicle behavior model neural network trained to generate predicted environment observations;
• generating an immediate quality value including providing the predicted environment of the autonomous vehicle and the candidate action as input to a context-specific quality model that generates immediate quality values that are specific to the particular driving context;
• generating an experience tuple comprising the initial environment observation, the candidate action, and the immediate quality value;
• and
• providing the experience tuple as input to a reinforcement learning system for the autonomous vehicle.