System and method for embedding uncertainty estimation into deep-neural-network-based autonomous driving perception frameworks

A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for training a perception model to perform an autonomous driving task, the method comprising:

• obtaining labeled training data comprising images captured by multiple cameras mounted at different locations on a vehicle;
• generating, in parallel by the perception model based on the labeled training data, a prediction output associated with the task and a confidence score, wherein the confidence score indicates a level of uncertainty correlated with visibility of the cameras and associated with the prediction output;
• generating an uncertainty-weighted prediction based on ground truth indicated by the labeled training data, the prediction output, and the confidence score, wherein generating the uncertainty-weighted prediction comprises computing a weighted combination of the prediction output and the ground truth, and wherein the prediction output is scaled by the confidence score and the ground truth is scaled by complement of the confidence score;
• computing a loss function based on the uncertainty-weighted prediction;
• and
• updating the perception model based on the loss function.

A computing system, comprising:

• a processor;
• and
• a memory coupled to the processor and storing instructions that when executed by the processor cause the processor to perform a method for training a perception model to perform an autonomous driving task, the method comprising:
• obtaining labeled training data comprising images captured by multiple cameras mounted at different locations on a vehicle;
• generating, in parallel by the perception model based on the labeled training data, a prediction output associated with the task and a confidence score, wherein the confidence score indicates a level of uncertainty correlated with visibility of the cameras and associated with the prediction output;
• generating an uncertainty-weighted prediction based on ground truth indicated by the labeled training data, the prediction output, and the confidence score, wherein generating the uncertainty-weighted prediction comprises computing a weighted combination of the prediction output and the ground truth, and wherein the prediction output is scaled by the confidence score and the ground truth is scaled by complement of the confidence score;
• computing a loss function based on the uncertainty-weighted prediction;
• and
• updating the perception model based on the loss function.