Polarized image enhancement using deep neural networks

A computing device for processing a polarized image, comprising:

• one or more processors;
• and
• data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to perform operations comprising:
• receiving, by the computing device, the polarized image from a polarization sensor configured to collect incident light at a plurality of polarization directions;
• determining, by the computing device, a polarization characterization for the polarized image, wherein the polarization characterization is indicative of polarization data associated with the plurality of polarization directions of the incident light in the polarized image;
• extracting, by the computing device and from the polarized image, a first collection of global features and a second collection of local features;
• performing, by the computing device and based on the polarization characterization:
• a global feature fusion to fuse global features in the first collection, and
• a local feature fusion to fuse local features in the second collection;
• compositing, by the computing device, the polarization characterization with the fused global features and the fused local features to generate a reconstructed image;
• providing, by the computing device, the reconstructed image to an image processing resource to perform one or more image processing tasks;
• and
• recommending, based on the reconstructed image, an adjustment to an automatic image capture setting for a camera associated with the polarization sensor.

A method for processing a polarized image, comprising:

• determining, by a computing device, a polarization characterization for a polarized image, wherein the polarization characterization is indicative of polarization data associated with a plurality of polarization directions of incident light in the polarized image;
• extracting, by the computing device and from the polarized image, a first collection of global features and a second collection of local features;
• performing, by the computing device and based on the polarization characterization:
• a global feature fusion to fuse global features in the first collection, and
• a local feature fusion to fuse local features in the second collection by utilizing a neural network to adaptively fuse an image intensity associated with the polarized image and the polarization characterization for the polarized image, and further comprising:
• training the neural network with training data adapted to one or more image processing tasks;
• compositing, by the computing device, the polarization characterization with the fused global features and the fused local features to generate a reconstructed image;
• and
• providing, by the computing device, the reconstructed image to an image processing resource to perform the one or more image processing tasks.