Jointly Learnable Behavior and Trajectory Planning for Autonomous Vehicles

A computer-implemented method of motion planning for an autonomous vehicle, the method comprising:

• obtaining, by a computing system comprising one or more computing devices, situational data associated with an environment detected by one or more sensors of the autonomous vehicle;
• generating, by the computing system using a behavioral planning stage and a unified cost function of a machine-learned motion planning system, data indicative of at least one behavioral decision for the autonomous vehicle based at least in part on the situational data;
• generating, by the computing system using a trajectory planning stage and the unified cost function of the machine-learned motion planning system, target trajectory data indicative of a target trajectory for the autonomous vehicle based at least in part on the data indicative of at least one behavioral decision for the autonomous vehicle;
• and
• generating, by the computing system, one or more motion plans based on the target trajectory.

A computing system, comprising:

• a machine-learned motion planning system configured to obtain situational data based at least in part on one or more outputs of a set of sensors of an autonomous vehicle and based at least in part on the situational data, generate a behavioral decision using a behavioral planning stage and output a target trajectory for the autonomous vehicle using a trajectory planning stage;
• one or more processors;
• and
• one or more non-transitory computer-readable media that store instructions, that when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
• inputting, to the machine-learned motion planning system, training data including annotated sensor data indicating ground truth vehicle trajectories;
• determining a loss associated with one or more target trajectories generated by the machine-learned motion planning system relative to the annotated sensor data based at least in part on a combined loss function including a first loss component associated with the behavioral planning stage and a second loss component associated with the trajectory planning stage;
• and
• backpropagating the loss associated with the one or more target trajectories to the machine-learned motion planning system to jointly train the behavioral planning stage and the trajectory planning stage.