Method for performing stereo matching to recover depths, colors and opacities of surface elements

A method for performing stereo matching comprising:

• selecting a general disparity space representing a projective sampling of 3D space, where the projective sampling includes an array of cells in the 3D space;
• transforming k input images from screen coordinates of corresponding input cameras to the general disparity space, where k is two or more and the input images are comprised of pixels each having color values;
• from the color values that map from the input images to the cells in the general disparity space, computing probabilities that the cells represent visible surface elements of an object depicted in the input images;
• computing initial estimates of color at the cells based on the probabilities;
• computing initial estimates of disparities of the visible surface elements based on the probabilities;
• computing initial estimates of opacities at the cells based on the probabilities;
• using the opacities to compute visibility values for the cells indicating whether the cells are visible with respect to the input cameras;
• and
• revising the initial color and disparity estimates based on the visibility values at the cells.

A method for performing stereo matching comprising:

• selecting a general disparity space representing a projective sampling of 3D space, where the projective sampling includes an array of cells at (x,y,d) coordinates in the 3D space, and d represents disparity planes comprised of (x,y) coordinates;
• transforming k input images from screen coordinates of corresponding input cameras to the general disparity space, where k is two or more and the input images are comprised of pixels each having color values;
• from the color values that map from the input images to the cells in the general disparity space, computing mean and variance of the color values at each cell;
• computing initial estimates of color at the cells based on the mean at each cell;
• computing initial estimates of disparities of visible surface elements based at least in part on the variance;
• computing initial estimates of opacities at the cells based at least in part on the variance at each cell;
• using the opacities to compute visibility values for the cells indicating whether the cells are visible with respect to the input cameras;
• revising the initial color and disparity estimates based on the visibility values;
• and
• refining estimates of color and opacity comprising:
• a) transforming current estimates of color and opacity from general disparity space to the (u,v,d) coordinate space of each of the input cameras;
• b) compositing the transformed, current estimates of color and opacity into re-projected images for each input camera;
• c) comparing the re-projected images with the input images to compute error values for each of the input cameras;
• d) adjusting the current estimates of color and opacity based on the error values;
• and
• e) repeating steps a-d to minimize the error values.

A method for performing stereo matching comprising:

• selecting a general disparity space representing a projective sampling of 3D space, where the projective sampling includes an array of cells in the 3D space;
• transforming k input images into the general disparity space, where k is two or more and the input images are comprised of pixels each having color values;
• estimating colors for the cells in the general disparity space based on the color values of the pixels from the k input images that map to the cells in the general disparity space;
• computing probabilities that the cells represent visible surface elements of an object depicted in the input images;
• and
• estimating opacities for the cells based on the probabilities, wherein the opacities are estimated by assigning totally opaque opacity values to cells that have a color variance below a threshold, and assigning totally transparent opacity values to cells that have a color variance above the threshold.