Context aware optimization of planner for autonomous driving

A system for optimizing a planner model for autonomous driving, the system comprising:

• a processor;
• and
• memory including instructions that, when executing by the processor, cause the processor to perform the following steps:
• receiving image data generated from a camera mounted to a vehicle, wherein the image data includes agents in an environment outside the vehicle;
• via a pre-existing perception model, detecting the agents in the environment based on the image data;
• executing a planner model to generate a predicted trajectory of the vehicle, wherein the generated predicted trajectory of the vehicle is dependent upon a set of configuration parameters associated with characteristics of the vehicle;
• receiving, from the perception model, information regarding a context associated with the detected agents in the environment, wherein the context comprises both (i) predicted motion vectors for each detected agent in the environment and (ii) categorical road-type label regarding a type of road;
• and
• executing a black-box optimizer configured to optimize performance of the planner model based on the context, wherein execution of the optimizer:
• (1) selects a subset of the configuration parameters to be optimized based on the context,
• (2) selects an objective function based on the context,
• (3) adjusts the selected subset of the configuration parameters based on a current value of the objective function,
• (4) generates a new planned trajectory of the vehicle based on the adjusted configuration parameters,
• (5) derives a value of the objective function based on the new planned trajectory, and
• (6) repeats (3)-(5) to determine optimal configuration parameters that minimize the objective function.

A non-transitory computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform:

• receive image data generated from a camera mounted to a vehicle, wherein the image data includes agents in an environment outside the vehicle;
• via a pre-existing perception model, detect the agents in the environment based on the image data;
• execute a planner model to generate a predicted trajectory of the vehicle, wherein the generated predicted trajectory of the vehicle is dependent upon a set of configuration parameters associated with characteristics of the vehicle;
• receive, from the perception model, information regarding a context associated with the detected agents in the environment, wherein the context comprises both (i) predicted motion vectors for each detected agent in the environment and (ii) categorical road-type label regarding a type of road;
• and
• execute a Bayesian optimizer configured to optimize performance of the planner model based on the context, wherein execution of the optimizer:
• (1) selects a subset of the configuration parameters to be optimized based on the context,
• (2) selects an objective function based on the context,
• (3) adjusts the selected subset of the configuration parameters based on a current value of the objective function,
• (4) generates a new planned trajectory of the vehicle based on the adjusted configuration parameters,
• (5) derives a value of the objective function based on the new planned trajectory, and
• (6) repeats (3)-(5) to determine optimal configuration parameters that minimize the objective function.