Method and system for training reinforcement learning agent using adversarial sampling

A system for training a reinforcement learning (RL) agent for autonomous operation of a vehicle, the system comprising:

• a memory;
• a processor coupled to the memory, the processor configured to execute instructions stored in the memory, to cause the system to:
• train the RL agent by:
• obtaining training samples by uniformly sampling a state space including all possible states of the vehicle and the environment;
• and
• learning a policy to map sampled states to actions outputted by the RL agent, the policy being learned by updating parameters of a neural network;
• determine that the RL agent has achieved a predetermined performance goal;
• collect data, collected data including a sequence of sampled states, the collected data further including, for each sequence of sampled states, agent parameters, and an indication of failure of the RL agent for the sequence;
• train, using samples from the collected data, a failure predictor to predict a probability of failure of the RL agent for a given sequence of states;
• simulate interaction of the vehicle with the environment to collect a plurality of sequences of states;
• select using the probability of failure outputted by the failure predictor, in a stochastic sampling method, a selected sequence of states from among the plurality of sequences of states;
• and
• further train the RL agent based on the selected sequence of states.

A computer-readable medium storing instructions, the instructions, when executed by a processor, causing a processor to train a reinforcement learning (RL) agent for autonomous operation of a vehicle by:

• training the RL agent by:
• obtaining training samples by uniformly sampling a state space including all possible states of the vehicle and the environment;
• and
• learning a policy to map sampled states to actions outputted by the RL agent, the policy being learned by updating parameters of a neural network;
• determining that the RL agent has achieved a predetermined performance goal;
• collecting data, collected data including a sequence of sampled states, the collected data further including, for each sequence of sampled states, agent parameters, and an indication of failure of the RL agent for the sequence;
• training, using samples from the collected data, a failure predictor to predict a probability of failure of the RL agent for a given sequence of states;
• simulating interaction of the vehicle with the environment to collect a plurality of sequences of states;
• selecting, using the probability of failure outputted by the failure predictor, in a stochastic sampling method, a selected sequence of states from among the plurality of sequences of states;
• and
• further training the RL agent based on the selected sequence of states.