Stereo depth estimation using deep neural networks

A processor comprising:

• processing circuitry to determine depth information using disparity maps computed using a neural network, wherein the neural network is trained based at least in part on:
• computing, using the neural network and based at least in part on a first image and a second image captured at a same time using a stereo camera, a first disparity map corresponding to the first image and a second disparity map corresponding to the second image;
• computing a first loss based at least in part on comparing the first image and the second image using at least one of the first disparity map or the second disparity map;
• receiving LIDAR data generated by at least one LIDAR sensor during the same time;
• computing a second loss based at least in part on comparing depth values determined from the LIDAR data to predicted depth values determined using at least one of the first disparity map or the second disparity map;
• and
• updating one or more parameters of the neural network based at least in part on the first loss and the second loss.

A system comprising:

• a stereo camera;
• one or more processing units;
• and
• one or more memory units storing instructions thereon that, when executed using the one or more processing units, cause the one or more processing units to execute operations comprising:
• computing, using a neural network and based at least in part on image data generated using the stereo camera, one or more disparity maps;
• and
• determining depth information based at least in part on the one or more disparity maps,
• wherein the neural network is trained, at least in part, by:
• computing, using the neural network and based at least in part on a first image and a second image captured at a same time, a first disparity map corresponding to the first image and a second disparity map corresponding to the second image;
• computing a first loss based at least in part on comparing the first image and the second image using at least one of the first disparity map or the second disparity map;
• receiving LIDAR data generated by at least one LIDAR sensor during the same time;
• computing a second loss based at least in part on comparing depth values determined from the LIDAR data to predicted depth values determined using at least one of the first disparity map or the second disparity map;
• and
• updating one or more parameters of the neural network based at least in part on the first loss and the second loss.