Systems and methods for dynamic refocusing of high resolution images generated using images captured by a plurality of imagers

A method of generating a high resolution image of a scene using an imager array including a plurality of imagers that each capture an image of the scene, and a forward imaging transformation for each imager, the method comprising:

• obtaining input images captured by a plurality of imagers using a processor configured by image processing pipeline software, where the input images capture a scene in which depths of points in the imaged scene vary and each of the input images differs from the other input images due to:
• scene independent geometric distortions inherent to the optics and manufacturing processes used to fabricate each of the plurality of imagers;
• and
• scene dependent geometric displacements due to parallax experienced by each of the plurality of imagers based upon the different depths of the points in the imaged scene;
• and
• determining scene dependent parallax information with respect to the input images based upon disparity relative to a reference point of view resulting from the different depths of points in the imaged scene using the processor configured by the image processing pipeline software, where the scene dependent parallax information comprises scene dependent geometric transformations;
• determining a total shift for each of a plurality of pixels relative to the reference point of view using the processor configured by the image processing pipeline software, where the total shift of a given pixel location is the combination of a scene independent geometric correction determined for the given pixel location using geometric calibration data and the scene dependent geometric transformation determined for the given pixel location;
• performing a super-resolution process utilizing at least a portion of plurality of input images and the total shift for each of the plurality of pixels relative to the reference point of view as inputs, where the super-resolution process comprises:
• determining an initial estimate of at least a portion of a high resolution image from a plurality of pixels from the input images using the processor configured by the image processing pipeline software based upon a total shift for each of the plurality of pixels relative to the reference point of view;
• determining a high resolution image that when mapped through a forward imaging transformation matches the input images to within at least one predetermined criterion using the processor configured using the image processing pipeline software based upon the initial estimate of at least a portion of the high resolution image, where:
• each forward imaging transformation corresponds to the manner in which each imager in the plurality of imagers captures the input images and comprises applying geometric transformations based upon the total shift at a given pixel location, which is the combination of a scene independent geometric correction determined for the given pixel location using geometric calibration data and the scene dependent geometric transformation determined for the given pixel location;
• and
• the high resolution image has a resolution that is greater than any of the input images;
• and
• generating a depth map for the high resolution image using a processor configured by image processing pipeline software by determining depth information for pixels in the high resolution image based upon the input images, parallax information, and the characteristics of the imager array;
• determining a focus map for the high resolution image using the depth map using a processor configured by image processing pipeline software, where the focus map identifies pixels having depths in the depth map that are within a specified depth of a defined focal plane;
• and
• performing dynamic refocus of the high-resolution image using a processor configured by image processing pipeline software by rendering the high resolution image using the focus map.