Imager for Detecting Visual Light and Projected Patterns

A method comprising:

• receiving, at a computing device and from a first plurality of photodetectors of a first optical sensor, a first visible light image of an environment as perceived from a first viewpoint;
• receiving, at the computing device and from a first plurality of photodetectors of a second optical sensor, a second visible light image of the environment as perceived from a second viewpoint, wherein the first optical sensor is separate from the second optical sensor;
• receiving, at the computing device and from a second plurality of photodetectors of the first optical sensor, a first infrared light image of the environment as perceived from the first viewpoint, wherein the second plurality of photodetectors of the first optical sensor is configured to capture infrared light within a particular infrared band;
• receiving, at the computing device and from a second plurality of photodetectors of the second optical sensor, a second infrared light image of the environment as perceived from the second viewpoint, wherein the second plurality of photodetectors of the second optical sensor is configured to capture infrared light within the particular infrared band;
• identifying, by the computing device, first corresponding features of the environment between the first visible light image and the second visible light image, wherein identifying the first corresponding features comprises determining a first correlation surface from the first visible light image and the second visible light image;
• identifying, by the computing device, second corresponding features of the environment between the first infrared light image and the second infrared light image, wherein identifying the second corresponding features comprises determining a second correlation surface from the first infrared light image and the second infrared light image;
• determining a combined correlation surface by combining the first correlation surface and the second correlation surface;
• and
• determining, by the computing device, a depth estimate for at least one surface in the environment based on the first corresponding features and the second corresponding features, wherein the depth estimate comprises a depth of a feature at a given pixel based on the combined correlation surface.

A non-transitory computer readable medium having stored therein program instructions executable by a computing system to cause the computing system to perform operations comprising:

• receiving, from a first plurality of photodetectors of a first optical sensor, a first visible light image of an environment as perceived from a first viewpoint;
• receiving, from a first plurality of photodetectors of a second optical sensor, a second visible light image of the environment as perceived from a second viewpoint, wherein the first optical sensor is separate from the second optical sensor;
• receiving, from a second plurality of photodetectors of the first optical sensor, a first infrared light image of the environment as perceived from the first viewpoint, wherein the second plurality of photodetectors of the first optical sensor is configured to capture infrared light within a particular infrared band;
• receiving, from a second plurality of photodetectors of the second optical sensor, a second infrared light image of the environment as perceived from the second viewpoint, wherein the second plurality of photodetectors of the second optical sensor is configured to capture infrared light within the particular infrared band;
• identifying first corresponding features of the environment between the first visible light image and the second visible light image, wherein identifying the first corresponding features comprises determining a first correlation surface from the first visible light image and the second visible light image;
• identifying second corresponding features of the environment between the first infrared light image and the second infrared light image, wherein identifying the second corresponding features comprises determining a second correlation surface from the first infrared light image and the second infrared light image;
• determining a combined correlation surface by combining the first correlation surface and the second correlation surface;
• and
• determining a depth estimate for at least one surface in the environment based on the first corresponding features and the second corresponding features, wherein the depth estimate comprises a depth of a feature at a given pixel based on the combined correlation surface.