US 2021/0241421 A1 · SYSTEMS AND METHODS FOR MULTISPECTRAL IMAGE DEMOSAICKING USING DEEP PANCHROMATIC IMAGE GUIDED RESIDUAL INTERPOLATION

Abstract

Described herein are systems and embodiments for multispectral image demosaicking using deep panchromatic image guided residual interpolation. Embodiments of a ResNet-based deep learning model are disclosed to reconstruct the full-resolution panchromatic image from multispectral filter array (MSFA) mosaic image. In one or more embodiments, the reconstructed deep panchromatic image (DPI) is deployed as the guide to recover the full-resolution multispectral image using a two-pass guided residual interpolation methodology. Experiment results demonstrate that the disclosed method embodiments outperform some state-of-the-art conventional and deep learning demosaicking methods both qualitatively and quantitatively.

Claims22

Dependent claims

A computer-implemented method for demosaicking a multispectral image from a multispectral filter arrays (MSFA) sensor with multiple sub-bands, the method comprising:

using a deep neural network and the multispectral image to reconstruct a full resolution deep panchromatic image (DPI) of the multispectral image;
for each sub-band of the multispectral image, performing at least one guided residual interpolation based on the reconstructed DPI to obtain a demosaicked sub-band image;
and
forming a demosaicked multispectral image comprising the demosaicked sub-band images.

1 more element

A computer-implemented method for demosaicking a sub-band image in a multispectral image captured by a multispectral filter arrays (MSFA) sensor using one or more processors to cause steps to be performed comprising:

using a deep neural network and the multispectral image to reconstruct a full resolution deep panchromatic image (DPI) of the multispectral image;
subsampling the DPI relative to the sub-band to obtain a subsampled DPI;
obtaining a subsampled residual image from a difference between the sub-band image and the subsampled DPI;
using the DPI as a guide image in a guided interpolation on the subsampled residual image to obtain a demosaicked residual image;
and
adding the demosaicked residual image to the DPI to obtain a first-pass demosaicked sub-band image.

3 more elements

A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by one or more processors, causes the steps for demosaicking a multispectral image, from a multispectral filter arrays (MSFA) sensor, with multiple sub-bands to be performed comprising:

reconstructing, using a deep neural network, the multispectral image to a full resolution deep panchromatic image (DPI);
for each sub-band of the multispectral image, performing at least one guided residual interpolation based on the reconstructed DPI to obtain a demosaicked sub-band image;
and
forming a demosaicked multispectral image comprising the demosaicked sub-band images.

1 more element

Dependencies are inferred from the claim text.

Systems and methods for multispectral image demosaicking using deep panchromatic image guided residual interpolation

Life of this patent

Systems and methods for multispectral image demosaicking using deep panchromatic image guided residual interpolation

Life of this patent

Abstract

Claims22