# US 2013/0278801 A1 · Image Processing Device, Image Processing Method, Program, and Imaging Device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0278801 A1
- Document ID: US20130278801A1
- Status: Published application
- Kind: A1
- Application number: 13/929,423
- Filed: 2013-06-27
- Published: 2013-10-24
- Earliest claimed: 2007-06-25
- Assignees: Silicon Hive B.V.
- Applicants: Silicon Hive B.V.
- Inventors: Yasuhiro Sawada; XianJi Zhang; Takashi Masuda
- Web page: https://patentgrep.com/patent/US20130278801A1

## CPC

- H04N 23/12: with one sensor only
- G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
- H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values}
- H04N 25/134: {based on three different wavelength filter elements}
- H04N 25/611: Correction of chromatic aberration
- H04N 2209/046: Colour interpolation to calculate the missing colour values
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image processing device includes: a coordinate conversion unit (142) which calculates a corresponding sampling coordinate on a color mosaic image corresponding to a pixel position of a color image when a deformation process is performed, according to the pixel position of the color image; a sampling unit (143); a sampling unit (143) which interpolates-generates a pixel value in a sampling coordinate for each of color planes obtained by decomposing the color mosaic image; and a color generation unit (144) which generates a color image by synthesizing interpolation values of the respective color planes. Each pixel value of a color image subjected to a deformation process is obtained as a pixel value of the sampling coordinate from the color mosaic image by interpolation calculation, thereby realizing the color interpolation process for generating a color image from the color mosaic image and a deformation process of the color image by one interpolation calculation.

## Claims

### Claim 1

An image processing apparatus that generates a color image containing multiple color luminance information of each pixel using a color mosaic image that contains single color luminance information of each pixel obtained from single-chip imaging elements as a plurality of photoelectrical converted pixels, the image processing apparatus comprising:
a color plane separator unit that separates the color mosaic image into multiple color planes having the same color;
a coordinates conversion unit that calculates sampling coordinates for the color mosaic image from a pixel position generated from the color mosaic image using a coefficient of an expression of an image processing of the color mosaic image during execution of a color image process for the color image;
a sampling unit that interpolative-generates color luminance values of same color pixels in the color planes as calculated by the coordinates conversion unit from pixel position values of the sampling coordinates; and
a color image generation unit that synthesizes interpolated values of each of the color planes provided by the sampling unit.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1 with the coordinate conversion unit, with coefficients expressing the image processing, and wherein the coordinate conversion unit calculates a corresponding sampling coordinate on a color mosaic image corresponding to a pixel position of a color image when a deformation process is performed, according to the pixel position of the color image.

### Claim 3 (depends on 1)

The image processing apparatus according to claim 1, wherein the color mosaic image is obtained from a red (R), green (Gr, Gb) and blue (B) Bayer array color filter, with imaging elements corresponding to each pixel image position, wherein the color plane separator unit separates the two kinds of green pixels, Gr and Gb, having corners in contact respectively, into different color planes.

### Claim 4 (depends on 3)

The image processing apparatus according to claim 3 including a color generation unit that takes the two kinds of green pixels, Gr and Gb, and adds the pixel values generated from the sampling unit, respectively, to generate the color image into two distinct color planes.

### Claim 5 (depends on 3)

The image processing apparatus according to claim 3 including the two kinds of green pixels, Gr and Gb, separated on each color plane generated by the sampling unit, and wherein a pixel value difference is calculated, and wherein in accordance with the pixel value difference value the apparatus is further equipped with a pseudo-color determination unit to judge if a pseudo color is present.

### Claim 6 (depends on 1)

The image processing apparatus according to claim 1 wherein the sampling unit performs interpolative-generation of the pixel values for surrounding pixels in the color plane for the pixel values at the sampling coordinates.

### Claim 7 (depends on 2)

The image processing apparatus according to claim 2 including, for the image deformation, at least one of the set of operations consisting of: magnification, reduction, rotation or skew correction processing.

### Claim 8 (depends on 1)

The image processing apparatus according to claim 1 wherein the coordinate conversion unit, with coefficients expressing the image processing, uses differing by color plane coefficients from the multiple color planes separated by the color plane separator unit, and wherein when executing chromatic aberration correction on the color image pixel position generated by the color mosaic image the coordinate conversion unit uses the color mosaic sampling coordinates corresponding to the color image pixel position, and wherein calculation of the differing sampling coordinates is done for each color plane.

### Claim 9 (depends on 1)

The image processing apparatus according to claim 1 in which the coordinate conversion unit, with coefficients expressing the image processing, uses differing by color plane coefficients from the multiple color planes separated by the above color plane separator unit and when executing chromatic aberration correction and the image deformation of the color image pixel position generated by the color mosaic image uses the color mosaic sampling coordinates corresponding to the color image pixel position, and calculation of the differing sampling coordinates is performed for each color plane.

### Claim 10 (depends on 1)

The image processing apparatus according to claim 1 in which the coordinate conversion unit, with coefficients expressing the image processing, uses differing by color plane coefficients from the multiple color planes separated by the color plane separator unit and when executing chromatic aberration correction and using the distortion aberration coefficient determined by the focal point distance and distance to photographed object as set for the imaging optics which introduces the image subject light, the image deformation of the color image pixel position generated by the color mosaic image uses the color mosaic sampling coordinates corresponding to the color image pixel position and calculation of the differing sampling coordinates is done for each color plane.

### Claim 11 (depends on 8)

The image processing apparatus according to claim 8 in which a further coefficient setting unit is equipped for setting the deformation aberration coefficient for the corresponding focal point length and distance to photographed subject set for the imaging optics.

### Claim 12

A color image processing method that generates a color image having multiple color information for each pixel from differing multiple photo electrically converted colored light pixels from single-chip imaging elements using a color mosaic image containing all single color luminance information of each pixel, the method comprising:
a first step to separate the color mosaic image into multiple color planes having only the same color; and
a second step calculating the sampling coordinates for the color mosaic image from the pixel position generated from the color mosaic image using a coefficient of an expression of image processing of the color image, during execution of the color image process for the color image, and
wherein during said first step, for each of the plurality of color planes, the sampling coordinates calculated in the second step are subjected, for the pixel values within the color plane of the same color, an interpolation generation sampling step, and
a color generation step that generates the color image through synthesis of a color plane interpolative value second step.

### Claim 13 (depends on 12)

The image processing method according to claim 12, in which in the coordinates conversion step, with coefficients expressing the image processing, using the image deformation coefficient for the color image generated by the color mosaic image, from the color image pixel position, when the image deformation is executed, the sampling coordinates on the color mosaic image corresponding to the color image pixel position is calculated.

### Claim 14 (depends on 12)

The image processing method according to claim 12 in which in the coordinates conversion step, with coefficients expressing the image processing, using the differing values of each of the color planes separated in the color plane separation step as coefficients for the color image generated by the color mosaic image, from the color image pixel position, when the chromatic aberration correction is executed, the differing sampling coordinates on each color plane are calculated.

### Claim 15

A computer-readable medium including stored computer-executable instructions for performing a color plane separation method that takes the color mosaic image having all of each pixels single color luminance information obtained from the multiple color photo electrically converted pixels of the imaging elements and separates these into color planes, each plane having only the exact color,
the color mosaic image sampling coordinates conversion method using the coefficients that express the color image generated from the color mosaic image, from the color image pixel position when image processing of the pixel position is executed,
an interpolative generation sampling method of all the same color value pixels within the color plane by the coordinates conversion method for each of the multiple color planes separated by the color plane separation method, and
through synthesis of each color plane interpolative value with the sampling method a color generation method for the color image containing the respective color value for one pixel, including a program that can be read by computer for computer function operation.

### Claim 16 (depends on 15)

The computer readable medium according to claim 15 wherein the sampling coordinates for the color mosaic image can be calculated, with the color mosaic image sampling coordinates conversion method using image deformation coefficients that express the color image generated from the color mosaic image, from the color image pixel position when deformation image processing of a pixel position is executed.

### Claim 17 (depends on 15)

The computer readable medium according to claim 15 in which the sampling coordinates for the differing color planes are calculated, with the coordinate conversion method for color plane sampling coordinates conversion using the differing value coefficients of the multiple color planes separated by the color plane separation method, generated from the color mosaic image, from the color image pixel position when chromatic aberration correction processing of the pixel position is executed.

### Claim at position 18

18-26. (canceled)
