Systems and methods for estimating depth from projected texture using camera arrays

A system incorporating a camera array capable of estimating depth information using projected texture, comprising:

• an illumination system configured to project texture;
• a camera array comprising:
• a first sub-array comprising a first camera;
• and
• a second sub-array comprising a second camera;
• wherein the first sub-array and the second-sub array are located in complementary occlusion zones around the illumination system such that the first sub-array and second sub-array are distributed on opposite sides of the illumination system such that when a projected texture is occluded from cameras in the first sub-array, the projected texture is in a field of view of at least one camera in the second sub-array;
• and
• wherein at least one of the cameras in the array of cameras is a camera capable of acquiring color images;
• a processor;
• and
• memory containing an image processing application;
• wherein the image processing application directs the processor to:
• direct the illumination system to illuminate a scene with a projected texture;
• capture, using the camera array, a set of images of the scene illuminated with the projected texture, where the set of images includes images captured by the first and second cameras;
• generate a map of depth information by measuring disparity observed between images in the set of images;
• generate image data concerning natural texture of the scene based upon at least an image captured by the camera that is capable of capturing color images when the scene is not illuminated with projected texture;
• register the map of depth information with respect to the image data concerning natural texture of the scene captured by the camera capable of capturing color images;
• generate 3D image data using the image data concerning natural texture of the scene and the registered map of depth information;
• render an image for display using the 3D image data;
• and
• display the rendered image.

A system incorporating a camera array capable of estimating depth information using projected texture, comprising:

• an illumination system configured to project texture;
• a camera array comprising:
• a first sub-array comprising a first camera;
• and
• a second sub-array comprising a second camera;
• wherein the first sub-array and the second-sub array are located in complementary occlusion zones around the illumination system such that the first sub-array and second sub-array are distributed on opposite sides of the illumination system such that when a projected texture is occluded from cameras in the first sub-array, the projected texture is in a field of view of at least one camera in the second sub-array;
• and
• wherein at least one of the cameras in the array of cameras is a camera capable of acquiring color images;
• a processor;
• and
• memory containing an image processing application;
• wherein the image processing application directs the processor to:
• direct the illumination system to illuminate a scene with a projected texture;
• capture, using the camera array, a set of images of the scene illuminated with the projected texture, where the set of images includes images captured by the first and second cameras;
• generate a map of depth information by measuring disparity observed between images in the set of images of the scene;
• generate image data concerning natural texture of the scene based upon an image captured by the camera that is capable of capturing color images when the scene is not illuminated with projected texture;
• register the map of depth information with respect to the image data concerning natural texture of the scene captured by the camera capable of capturing color images;
• generate a mesh for at least a portion of the scene visible within the image captured by the camera that is capable of capturing color images, where the mesh is generated using the registered map of depth information;
• render an image for display using the mesh and the image data concerning natural texture of the scene;
• and
• display the rendered image.