Device and method for efficient inverse reinforcement learning for autonomous driving

A non-transitory machine-readable storage medium on which is stored a computer program for training a driving policy for an autonomous vehicle, the computer program, when executed by a processor, causing the processor to perform the following steps comprising:

• a) pre-training the driving policy in a first stage using one of the following techniques:
• i) Reinforcement Learning (RL) with a hand-crafted reward function, or
• ii) behavior cloning from expert demonstrations;
• b) storing intermediate results from the first stage, including at least one of:
• i) rollouts generated by the driving policy, or
• ii) model weights of the pre-trained driving policy;
• c) applying Inverse Reinforcement Learning (IRL), including at least one of:
• i) learning a reward model from expert demonstrations, or
• ii) fine-tuning the pre-trained driving policy from the first stage with RL under the learned reward model;
• d) using results of the first stage during a secondary stage, including:
• i) using the stored rollouts to augment the expert demonstration dataset, and
• ii) reusing the stored model weights to initialize the driving policy and the reward model.

A system configured to train a driving policy for an autonomous vehicle, the system configured to perform the following steps comprising:

• a) pre-training the driving policy in a first stage using one of the following techniques:
• i) Reinforcement Learning (RL) with a hand-crafted reward function, or
• ii) behavior cloning from expert demonstrations;
• b) storing intermediate results from the first stage, including at least one of:
• i) rollouts generated by the driving policy, or
• ii) model weights of the pre-trained driving policy;
• c) applying Inverse Reinforcement Learning (IRL), including at least one of:
• i) learning a reward model from expert demonstrations, or
• ii) fine-tuning the pre-trained driving policy from the first stage with RL under the learned reward model;
• d) using results of the first stage during a secondary stage, including:
• i) using the stored rollouts to augment the expert demonstration dataset, and
• ii) reusing the stored model weights to initialize the driving policy and the reward model.