US 8,577,184 B2Granted

System and method for super-resolution imaging from a sequence of color filter array (CFA) low-resolution images

AssigneeThe United States of America as represented by the Secretary of the Army

InventorShiqiong Susan Young

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method and system for improving picture quality of color images by combing the content of a plurality of frames of the same subject; comprising: at least one processor; the at least one processor comprising a memory for storing a plurality of frames of a subject; the at least one processor operating to combine the content of plurality of frames of the subject into a combined color image by performing: a process in which at least two multicolored frames are converted to monochromatic predetermined color frames; a gross shift process in which the gross shift translation of one monochromatic predetermined color frame is determined relative to a reference monochromatic predetermined color frame; a subpixel shift process utilizing a correlation method to determine the translational and/or rotational differences of one monochromatic predetermined color frame to the reference monochromatic predetermined color frame to estimate sub-pixel shifts and/or rotations between the frames; and an error reduction process to determine whether the resolution of the resulting combined color image is of sufficient resolution; the error reduction process comprising applying at least one spatial frequency domain constraint and at least one spatial domain constraint to the combined color image to produce at least one high-resolution full color image.

Claims22

Dependent claims

A system for improving the quality of color images by combining the content of a plurality of frames of the same subject comprising:

at least one processor;
the at least one processor operatively connected to a memory for storing a plurality of frames of a subject;
the frames containing imperfect images of the subject due being affected by translation and/or rotational movement;
the at least one processor operating to align and combine the content of plurality of frames of the subject into a combined multicolored image;
the at least one processor operating to convert at least two multicolored frames into monochromatic predetermined color frames;
the at least one processor operating to estimate the optimal orientation by using the at least two monochromatic frames and by estimating optimal alignment of the two monochromatic frames at the frame and subpixel levels;
at the frame level, the at least one processor operating to perform a gross shift process in which the gross shift translation of one monochromatic predetermined color frame is determined relative to another monochromatic predetermined color frame;
at the subpixel level, the at least one processor operating to perform a subpixel shift process utilizing a correlation method to determine the translational and/or rotational differences of one monochromatic predetermined color frame to the other monochromatic predetermined color frame to estimate sub-pixel shifts and/or rotations between the frames;
the at least one processor combining the at least two multicolored frames into a multicolored color image based upon the results of the monochromatic gross shift and subpixel shift processes;
and
checking to determine whether the resolution of the resulting combined multicolored image is of sufficient resolution;
the checking process operating to determine the validity of the gross shift and subpixel shift processes to produce at least one high-resolution multicolored image.

9 more elements

A system for super-resolving images from color filter array (CFA) low-resolution sequences comprising:

at least one processor having an input for inputting multiple CFA images of a scene with sub-pixel translations and rotations;
one of the images of the scene being a reference image;
and
at least one memory operatively connected to the at least one processor, the at least one processor operating to estimate the optimal orientation by first converting the multicolored frames to monochromatic frames using a monochromatic color image constructing algorithm for creating at least one monochromatic color image for each image in the input sequence;
the at least one processor operating to estimate the shift of the inputted CFA images using monochromatic frames and performing estimates of the amount of shift of the image from one image to a reference image at the frame and subpixel levels by using a gross translation estimation algorithm for estimating overall translations of the at least one monochromatic color image with respect to the reference image on the image processor;
a sub-pixel estimation algorithm for estimating the sub-pixel translations and/or rotations of at least one monochromatic color image with respect to the reference image;
and
the at least one processor operating to check and validate the estimates obtained using the gross translation and subpixel processes to produce at least one high-resolution full color image.

5 more elements

A method of super-resolving CFA images affected by movement including translation and rotation comprising:

inputting frames of low-resolution CFA images of a scene into a processor;
at least one of the frames being affected by movement;
obtaining at least one monochromatic color image for each image in the input frames;
estimating the gross translations and sub-pixel shifts and rotations due to the movement using the monochromatic color images;
and
checking the validity of the estimated gross translations and subpixel shifts to produce a high-resolution full color output image.

3 more elements

Dependencies are inferred from the claim text.

Facts

Application: 12/968,881Filed 15 Dec 2010 · continuation in part
Priority: 19 Jan 2005
Published: US 2011/0142366 A116 Jun 2011
Granted: 5 Nov 2013
Term ends: Not estimatedThe source records don’t support an estimate.

Classification2 codes

G06T 3/4069
Based on super-resolution, i.e. the output image resolution being higher than the sensor resolution: by subpixel displacements
G06T
Image data processing or generation, in general

Family

Application11/038,401Parent of this continuation-in-part · filed 19 Jan 2005
Application12/576,132Parent of this continuation-in-part · filed 8 Oct 2009
US 7,602,997 B2Parent of this continuation-in-part · filed 13 Oct 2009
US 2011/0142366 A1Pre-grant publication · published 16 Jun 2011
US 7,856,154 B2Parent of this continuation-in-part
US 2006/0159369 A1Published application
US 2010/0067822 A1Published application
US 8,577,184 B2This patent

From the US filing history. Foreign family members aren’t listed.