Image demosaicing and enhancement system

#The present invention relates to digital cameras, and more particularly, to an improved method for converting data from a camera sensor to a color image.

#A digital color image usually consists of an array of pixel values representing the intensity of the image at each point on a regular grid. Typically, three colors are used to generate the image. At each point on the grid the intensity of each of these colors is specified, thereby specifying both the intensity and color of the image at that grid point.

#Conventional color photography records the relevant image data by utilizing three overlapping color sensing layers having sensitivities in different regions of the spectrum (usually red, green, and blue). Digital cameras, in contrast, typically utilize one array of sensors in a single “layer”.

#FIG. 1 illustrates the separation of an image taken with an image sensor having a repeating 2×2 pattern into image planes according to the present invention.

#FIG. 2 illustrates a portion of a sensor array and the input pixels in the sensor array which contribute to a particular intermediate input vector.

#FIG. 3 illustrates a portion of an output RGB (red, green, blue) image and the pixels in the RGB output image that correspond to the intermediate output vector shown in FIG. 2.

#The method of the present invention may be applied to any color-sampling device that acquires its samples from a sensor array that can be decomposed into a plurality of image planes, which satisfy two conditions. First, each image plane must have an identical number of samples within a normalized horizontal and vertical sampling interval; however, the various image planes may be arbitrarily displaced relative to one another. Second, all samples in a given image plane must have identical color properties; however, multiple image planes can have the same color properties.

#These conditions are satisfied by any image sensor having a sensor pattern that is constructed by repeating a kernel of sensing elements. For example, one common image sensor array is based on the Bayer pattern which generated by repeating a 2×2 sensor array kernel having two green sensors, one red sensor, and one blue sensor. This pattern is shown in FIG. 1 at 10. The kernel is shown at 12. Such an image sensor may be viewed as having 4 planes shown at 14-17, two green planes 14 and 17, one red plane 16, and one blue plane 15. The sampling interval is the area originally occupied by the kernel. Each of the planes is offset with respect to the other planes. It can be shown that any regular sampling lattice can be decomposed into a set of image planes satisfying the above conditions.

#To simplify the following discussion, vector notation will be utilized. Vectors and matrices will be shown in bold print to distinguish them from scalar quantities. The measured intensity values in each image plane will be denoted by xp[n1,n2]. Here, n, and n2 are indicies which denote the position of the pixel in the pth image plane and xp is the intensity value measured for that pixel. The quantity [n1,n2] is a two-dimensional integer valued vector which will be denoted by n. The entire set of image planes can then be represented as a set of vectors x[n] where 1 x ⁡ [ n ] = [ x 1 ⁡ [ n 1 , n 2 ] x 2 ⁡ [ n 1 , n 2 ] ⋮ x P ⁡ [ n 1 , n 2 ] ] ( 1 )