Creating 3D images of objects by illuminating with infrared patterns

A system comprising:

• a first stereo camera pair including a first camera coupled to a second camera;
• a second stereo camera pair including a third camera coupled to a fourth camera;
• a set of four infra-red filters, with a separate one of the four infra-red filters operatively coupled to each of four cameras;
• a projector;
• and
• a computer readable medium coupled to each of the four cameras and to the projector, and including instructions for performing at least the following:
• projecting an infra-red pattern from the projector onto a three-dimensional object;
• producing a first image of the three-dimensional object while the pattern is projected on the three-dimensional object, the first image (i) including the three-dimensional object and the pattern, (ii) being a two-dimensional digital image including pixels, and (iii) being produced by capturing at a first camera light filtered through an infra-red filter;
• producing a second image of the three-dimensional object while the pattern is projected on the three-dimensional object, the second image (i) including the three-dimensional object and the pattern, (ii) being a two-dimensional digital image including pixels, and (iii) being produced by capturing at a second camera light filtered through an infra-red filter, and the first and second cameras arranged as a first stereo pair having a known physical relationship;
• identifying, based on a deformation of the pattern, a depth discontinuity occurring at a junction of surfaces of the three-dimensional object having different depths;
• establishing a first-pair correspondence between a portion of the pixels of the first image and a portion of the pixels of the second image;
• controlling, based on the identified depth discontinuity, a disparity propagation of the portion of the pixels of the first image and the portion of the pixels of the second image established in the first-pair correspondence;
• producing a third image of the three-dimensional object while the pattern is projected on the three-dimensional object, the third image (i) including the three-dimensional object but not the pattern, (ii) being a two-dimensional digital image including pixels, and (iii) being produced by capturing light at a texture camera;
• and
• constructing, based on the disparity propagation of the portion of the pixels of the first image and the portion of the pixels of the second image established in the first-pair correspondence and the third image, a two-dimensional image that depicts a three-dimensional construction of the three-dimensional object.

A computer readable medium including instructions for performing at least the following:

• accessing a first image captured of a three-dimensional object, the first image having been captured while a pattern is projected on the three-dimensional object, the first image (i) including the three-dimensional object and the pattern, (ii) being a two-dimensional digital image including pixels, and (iii) being produced by capturing at a first camera light filtered through an infra-red filter;
• accessing a second image captured of the three-dimensional object, the second image having been captured while the pattern is projected on the three-dimensional object, the second image (i) including the three-dimensional object and the pattern, (ii) being a two-dimensional digital image including pixels, and (iii) being produced by capturing at a second camera light filtered through an infra-red filter;
• identifying, based on a deformation of the pattern, a depth discontinuity occurring at a junction of surfaces of the three-dimensional object having different depths;
• establishing a first-pair correspondence between a portion of the pixels of the first image and a portion of the pixels of the second image, the first-pair correspondence being established based on the first and second cameras having been arranged as a first stereo pair having a known physical relationship while the first and second images were captured;
• controlling, based on the identified depth discontinuity, a disparity propagation of the portion of the pixels of the first image and the portion of the pixels of the second image established in the first-pair correspondence;
• accessing a third image captured of the three-dimensional object, the third image having been captured while the pattern is projected on the three-dimensional object, the third image (i) including the three-dimensional object but not the pattern, (ii) being a two-dimensional digital image including pixels;
• and
• constructing, based on the disparity propagation of the portion of the pixels of the first image and the portion of the pixels of the second image established in the first-pair correspondence and the third image, a two-dimensional image that depicts a three-dimensional construction of the three-dimensional object.

A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

• generate first and second images of an object illuminated with a pattern;
• establish a first-pair correspondence between initial matched pixels of the first and second image based on detecting the illuminated pattern in the first and second images;
• identify a depth discontinuity associated with the object based on a deformation of the illuminated pattern;
• control a disparity propagation of the initial matched pixels in two directions based on the identified depth discontinuity;
• and
• generate a three-dimensional reconstruction of the object based on the controlled disparity propagation.

A computer program product, tangibly embodied in a machine readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

• receive first and second pairs of images in a sequence of paired images of an object illuminated with a pattern;
• and
• for each of the first and second pairs of images:
• establish a first-pair correspondence between initial matched pixels of the paired images based on detecting the illuminated pattern in the paired images,
• identify a depth discontinuity associated with the object based on a deformation of the illuminated pattern,
• control a disparity propagation of the initial matched pixels in two directions based on the identified depth discontinuity, and
• generate a three-dimensional reconstruction of the object based on the controlled disparity propagation.

A computer program product, tangibly embodied in a machine readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

• receive first and second images of an object illuminated with a pattern of uncoded stripes;
• establish a first-pair correspondence between initial matched pixels of the paired images based on detecting the illuminated pattern in the paired images;
• identify a depth discontinuity associated with the object based on a deformation of the illuminated pattern;
• control a disparity propagation of the initial matched pixels in two directions based on the identified depth discontinuity;
• and
• generate a three-dimensional reconstruction of the object based on the controlled disparity propagation.