Sensor fusion for depth estimation

A computer program product for calculating a depth value for a pixel in a reference image, the computer program product comprising:

• a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code configured to:
• receive the reference image captured by a reference camera and at least one auxiliary image captured by an auxiliary camera;
• receive a first support value indicating whether the pixel in the reference image is at a particular depth, relative to the reference camera, based on comparing a region of the auxiliary image with a region of the reference image;
• provide a depth estimate of the pixel from a range-estimation camera;
• receive a second support value indicating whether the pixel in the reference image is at the particular depth based on comparing the depth estimate from the range-estimation camera to the particular depth;
• receive a third support value indicating whether the pixel is at the particular depth based on projecting a 3D point, corresponding to the pixel in the reference image, onto the auxiliary image;
• and
• fuse the first, second, and third support values to generate a total support value for the pixel at the particular depth.

A system, comprising:

• a common support structure comprising:
• a reference camera,
• at least one auxiliary camera, and
• a range-estimation camera;
• and
• a computing device communicatively coupled to the reference, auxiliary, and ToF cameras, the computing device configured to:
• calculate a first support value indicating whether a pixel in a reference image captured by the reference camera is at a particular depth, relative to the reference camera, based on comparing a region of the reference image with a region of an auxiliary image captured by the auxiliary camera;
• calculate a second support value indicating whether the pixel in the reference image is at the particular depth based on comparing a depth estimate from the range-estimation camera to the particular depth;
• calculate a third support value indicating whether the pixel is at the particular depth based on projecting a 3D point, corresponding to the pixel in the reference image, onto the auxiliary image;
• and
• fuse the first, second, and third support values to generate a total support value for the pixel at the particular depth.

A method for adjusting a depth value for a pixel in an image, the method comprising:

• providing a depth map associated with the image, wherein the depth map comprises respective depth values for a plurality of pixels in the image;
• grouping a subset of the plurality of pixels into a super pixel based on comparing respective thermal values associated with neighboring pixels;
• and
• estimating a depth plane for the super pixel based on the respective depth values of the subset of the plurality of pixels;
• and
• adjusting at least one of the respective depth values by comparing the respective depth values of the subset of the plurality of pixels to the estimated depth plane.