Auto-tuning motion planning system for autonomous vehicles

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

• collecting information for a driving environment surrounding the ADV using a plurality of sensors of the ADV;
• generating a plurality of sample trajectories from a trajectory sample space for the driving environment;
• determining a reward based on a reward model for each of the sample trajectories, wherein the reward model is generated using a rank based conditional inverse reinforcement learning algorithm, wherein the rank based conditional inverse reinforcement learning algorithm is a rank based inverse reinforcement learning conditional on a driving scenario such that the conditional inverse reinforcement learning algorithm is trainable scenario-wise, wherein the driving scenario includes a frame of a planning cycle;
• wherein the reward model is also generated by:
• generating a Siamese network for the reward model based on the plurality of sample trajectories in the trajectory sampling space and a target trajectory, wherein the target trajectory is an expert trajectory;
• and
• applying an inverse reinforcement learning algorithm to the Siamese network to determine one or more weighting factors for the reward model to place the target trajectory in a highest ranking among the plurality of sample trajectories, wherein each of the weighting factors correspond to a respective feature for the reward model;
• ranking the sample trajectories based on the determined rewards;
• determining a highest ranked trajectory based on the ranking, from the sample trajectories based on the ranking;
• and
• controlling the ADV autonomously according to the highest ranked trajectory.

A computer-implemented method to train a rewards model for an autonomous driving vehicle (ADV), the method comprising:

• determining a target trajectory based on driven trajectories collected from one or more vehicles;
• generating a plurality of sample trajectories from a trajectory sample space for a driving environment of the target trajectory;
• and
• generating a reward model by applying a rank based conditional inverse reinforcement learning algorithm to the sample trajectories and the target trajectory, wherein the reward model is used by an ADV to generate a driving trajectory to control the ADV, wherein the rank based conditional inverse reinforcement learning algorithm is a rank based inverse reinforcement learning conditional on a driving scenario such that the conditional inverse reinforcement learning algorithm is trainable scenario-wise, wherein the driving scenario includes a frame of a planning cycle;
• wherein generating a reward model also includes applying a rank based conditional inverse reinforcement learning algorithm comprises:
• generating a Siamese network for the reward model based on the plurality of sample trajectories in the trajectory sampling space and a target trajectory;
• and
• applying an inverse reinforcement learning algorithm to the Siamese network to determine one or more weighting factors for the reward model to place the target trajectory in a highest ranking among the plurality of sample trajectories, wherein each of the weighting factors correspond to a respective feature for the reward model, and
• ranking the sample trajectories based on the determined rewards;
• determining a highest ranked trajectory based on the ranking, from the sample trajectories based on the ranking;
• and
• controlling the ADV autonomously according to the highest ranked trajectory.