Occlusion handling in stereo imaging

A computing system, comprising:

• one or more processors;
• a memory device, configured for access by the one or more processors;
• and
• processor executable instructions, defined on the memory device and executable by the one or more processors, for:
• segmenting a first stereo image to result in a coarse segmentation;
• using the coarse segmentation to define a space within which an optimal solution exists;
• segmenting the first stereo image and a second stereo image to result in fine segmentations;
• mapping discontinuities within one of the fine segmentations to discover disparities and occlusions in the other fine segmentation;
• optimizing energy of a current solution within the defined space, wherein the current solution comprises the discovered disparities and occlusions;
• pairing two fine segmentations, the pairing performed by:
• finding a first fine segmentation within the first stereo image and a second fine segmentation within the second stereo image, wherein a sum of absolute difference (SAD) error between the first and second fine segmentations is minimized;
• calculating a measure of smoothness energy between the second fine segmentation and at least one neighbor of the second fine segmentation on the second stereo image, wherein the measure of smoothness energy is based on a difference between labels of two neighboring segments;
• and
• evaluating the SAD error and the measure of smoothness energy to determine if the first fine segmentation within the first stereo image should be paired with the second fine segmentation within the second stereo image;
• and
• obtaining disparities and occlusions of pixels in the pair of segmentations within the stereo images from the energy optimization.

A computer implemented method for occlusion handling in stereo imaging, comprising:

• segmenting a first stereo image to result in a coarse segmentation, the first stereo image being either a left or right image;
• labeling the coarse segmentation of the first stereo image;
• setting an initial estimate of a disparity and occlusions between the first stereo image and a second stereo image, the initial estimate based on the labeling of the coarse segmentation, the second stereo image being the other of the left or the right image;
• segmenting the first and second stereo images to result in fine segmentations, the fine segmentations being more granular than the coarse segmentation;
• mapping a portion of a border of a discontinuity in the second stereo image to indicate an occlusion in the first stereo image;
• computing an energy level of a current solution comprising occlusions and disparities determined by the mapping, the computing of the energy level based on the initial estimate and the fine segmentations;
• optimizing the energy level, initially based on the initial estimate, and thereafter using an α-expansion framework;
• pairing two fine segmentations, the pairing performed by:
• finding a first fine segmentation within the first stereo image and a second fine segmentation within the second stereo image, wherein a sum of absolute difference (SAD) error between the first and second fine segmentations is minimized;
• calculating a measure of smoothness energy between the second fine segmentation and at least one neighbor of the second fine segmentation on the second stereo image, wherein the measure of smoothness energy is based on a difference between labels of two neighboring segments;
• and
• evaluating the SAD error and the measure of smoothness energy to determine if the first fine segmentation within the first stereo image should be paired with the second fine segmentation within the second stereo image;
• and
• obtaining disparities and occlusions of pixels in the paired fine segmentations based on the optimized energy level.