Apparatus and methods for three-dimensional sensing

A method for sensing depth, the method comprising:

• alternatively, capturing by two dual band cameras a first set of frames and a second set of frames, the first set of frames forming a stereo pair captured with a pattern IR illuminator in an off state and the second set of frames having stereo pairs captured with the pattern IR illuminator in an on state;
• generating a disparate stream from the second set of frames;
• performing stereo to the second set of frames by:
• selecting a point in a first frame of the second set of frames;
• computing, by a computer processor, a matching score along an epipolar line in a second frame of the second set of frames;
• minimizing the matching score to thereby identify disparity;
• and
• performing a shift-to-depth based on geometric and optical properties of the second set of frames.

A method for sensing depth, the method comprising:

• capturing a first image by a near-infrared (NIR) camera of an image device;
• substantially simultaneously with capturing the first image, capturing a second image by an ultraviolet (UV) camera of the image device, the NIR camera and the UV camera aligned horizontally along the image device with an IR emitter of the image device and the first image and second image captured while the IR emitter is in an on state such that the first image and second image form a pattern stereo pair;
• capturing a third image the NIR camera of an image device;
• substantially simultaneously with capturing the third image, capturing a fourth image by the UV camera while the IR emitter is in an off state such that the third image and forth image form a visible stereo pair;
• generating a first disparate stream from the first image and second image;
• generating a second disparate stream from the third image and fourth image;
• performing stereo depth computation on the first disparate stream by:
• selecting a pixel in the first image;
• determining an epipolar line between the first image and the second image;
• computing, by a computer processor, a matching score of a pixel in the second image along the epipolar line;
• identifying a disparity between the first image and the second image based at least in part on the matching score;
• and
• performing a shift-to-depth based on geometric and optical properties of the first image and the second image in response to identifying the disparity.

A method of performing depth sensing using multi-purpose camera modules, the method comprising:

• using at least two dual band near-infrared (NIR) or ultraviolet (UV) cameras to simultaneously capture one or more stereo pairs, wherein the at least two dual band NIR or UV cameras are configured to capture images using visible light and NIR or UV;
• alternating between two types of stereo pairs using the at least two dual band NIR or UV cameras by:
• capturing a first stereo pair with a pattern IR illuminator in an off state (a “visible stereo pair”), and
• capturing a second stereo pair with the pattern IR illuminator in an on state (a “pattern stereo pair”);
• generating two disparate streams produced by steps (i)-(ii);
• and
• performing stereo depth computation on frames captured with the pattern IR illuminator in the on state by:
• (a) selecting a point in one image of the simultaneously captured one or more images,
• (b) computing, by a computer processor, a matching score along an epipolar line in another of the simultaneously captured images,
• (c) minimizing the matching score to thereby identify disparity, and
• (d) performing a shift-to-depth based on geometric and optical properties of the one image and the another image of the simultaneously captured images.

A method of performing depth sensing using multi-purpose camera modules, the method comprising:

• using at least two dual band cameras to simultaneously capture a plurality of frames, each of the plurality of frames including stereo pair of images, wherein the at least two dual band cameras are configured to capture images using visible light and near-infrared (NIR);
• alternatively, capturing a first set of frames and a second set of frames, the first set of frames having a stereo pair captured with a pattern IR illuminator in an off state and the second set of frames having a stereo pairs captured with the pattern IR illuminator in an on state;
• generating a first disparate stream from the first set of frames;
• generating a second disparate stream from the second set of frames;
• performing stereo depth computation on the second set of frames by:
• selecting a point in a first image of individual stereo pairs of the second set of frames, computing, by a computer processor, a matching score along an epipolar line in a second image of the individual stereo pairs of the second set of frames,
• reducing disparities between the first image and the second image of the individual stereo pairs of the second set of frames based at least in part on the matching score, and
• performing a shift-to-depth based on geometric and optical properties of the first image and the second image of the individual stereo pairs of the second set of frames.