Calibrating texture cameras using features extracted from depth images

A method for calibrating texture cameras, comprising:

• receiving, via a processor, a plurality of depth images and a plurality of color images from a depth camera system;
• extracting, via the processor, features from the plurality of depth images and the plurality of color images;
• detecting, via the processor, that the extracted features from the plurality of depth images and the plurality of color images exceed a quality threshold based on a smoothness of depth edges;
• aggregating, via the processor, the extracted features from the plurality of depth images to generate a first aggregated set of extracted features and aggregating the extracted features from the plurality of color images to generate a second aggregated set of extracted features;
• detecting, via the processor, that a misalignment between the first aggregated set of extracted features from the plurality of depth images and the second aggregated set of extracted features from the plurality of color images exceeds a misalignment threshold;
• and
• modifying, via the processor, calibration parameters for a texture camera to reduce the detected misalignment between the first aggregated set of extracted features from the plurality of depth images and the second aggregated set of extracted features from the plurality of color images below the misalignment threshold,
• wherein the calibration parameters are modified by solving an optimization problem comprising the calibration parameters and features extracted from a texture mapping of a color image among the plurality of color images mapped to a depth image among the plurality of depth images.

At least one non-transitory computer readable medium for calibrating texture cameras in depth camera systems having instructions stored therein that, in response to being executed on a computing device, cause the computing device to:

• receive a plurality of depth images from an infrared camera and a plurality of color images from a texture camera of a depth camera system;
• extract features from the plurality of depth images and the plurality of color images;
• detect that the extracted features from the plurality of depth images and the plurality of color images exceed a quality threshold based on a smoothness of depth edges;
• aggregate the extracted features from the plurality of depth images to generate a first aggregated set of extracted features and aggregate the extracted features from the plurality of color images to generate a second aggregated set of extracted features;
• detect that a misalignment between the first aggregated set of extracted features from the plurality of depth images and the second aggregated set of extracted features from the plurality of color images exceeds a misalignment threshold;
• and
• modify calibration parameters for the texture camera to reduce the detected misalignment between the first aggregated set of extracted features from the plurality of depth images and the second aggregated set of extracted features from the plurality of color images below the misalignment threshold,
• wherein the calibration parameters are modified by solving an optimization problem comprising the calibration parameters and features extracted from a texture mapping of a color image among the plurality of color images mapped to a depth image among the plurality of depth images.