Unified deep convolutional neural net for free-space estimation, object detection and object pose estimation

Non-transient computer readable media encoded with computer programming instructions configurable to cause one or more processors to perform a method in a vehicle of performing multiple on-board sensing tasks concurrently in the same network using deep learning algorithms, the method comprising:

• receiving vision sensor data from a sensor on the vehicle;
• determining a set of features from the vision sensor data using a plurality of feature layers in a convolutional neural network, the convolutional neural network comprising:
• a plurality of free-space estimation layers configured to evaluate the set of features to determine the boundaries of free-space in the vision sensor data relative to the vehicle and mark the boundaries;
• a plurality of object detection layers configured to evaluate the set of features to detect objects in the image and to estimate bounding boxes that surround the detected objects;
• and
• a plurality of object pose detection layers configured to evaluate the set of features to estimate the direction of each object, wherein the object pose detection layers have been trained using a cyclic piecewise linear (PL) loss function, the training comprising:
• assigning a different value between zero and 360 degrees to each of a plurality of bins;
• assigning, with a weighting factor, an estimated pose to the two bins of the plurality of bins with assigned center values that are closer to the estimated pose value wherein the weighting factor is inversely proportional to the distance of the estimated pose value from the bin center value;
• and
• assigning an estimated pose to both the highest value bin and the lowest value bin if the estimated pose value is higher than the value assigned to the highest value bin or if the estimated pose value is lower than the value assigned to the lowest value bin;
• and
• concurrently estimating, using the convolutional neural network, bounding boxes for detected objects, free-space boundaries, and object poses for detected objects from the set of features determined by the plurality of feature layers.

A processor-implemented sensing system in a vehicle for concurrently performing free-space estimation, object detection, and object pose estimation concurrently on vision sensor data, the sensing system comprising one or more processors and non-transient computer readable media encoded with programming instructions configurable to cause the one or more processors to perform a method, the method comprising:

• receiving vision sensor data from a sensor on the vehicle;
• determining a set of features from the vision sensor data using a plurality of feature layers in a convolutional neural network;
• and
• concurrently estimating, using the convolutional neural network:
• bounding boxes for detected objects using a plurality of object detection layers in the convolutional neural network configured to evaluate the set of features to detect objects in the image and to estimate bounding boxes that surround the detected objects;
• free-space boundaries using a plurality of free-space estimation layers in the convolutional neural network configured to evaluate the set of features to determine the boundaries of free-space in the vision sensor data relative to the vehicle and mark the boundaries;
• and
• object poses for detected objects using a plurality of object pose detection layers in the convolutional neural network configured to evaluate the set of features to estimate the direction of each object, wherein the object pose detection layers have been trained using a cyclic piecewise linear (PL) loss function, the training comprising:
• assigning a different value between zero and 360 degrees to each of a plurality of bins;
• assigning, with a weighting factor, an estimated pose to the two bins of the plurality of bins with assigned center values that are closer to the estimated pose value wherein the weighting factor is inversely proportional to the distance of the estimated pose value from the bin center value;
• and
• assigning an estimated pose to both the highest value bin and the lowest value bin if the estimated pose value is higher than the value assigned to the highest value bin or if the estimated pose value is lower than the value assigned to the lowest value bin.