# US 8,571,312 B2 · Image interpolation method and apparatus using pattern characteristics of color filter array

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,571,312 B2
- Document ID: US8571312B2
- Status: Granted
- Kind: B2
- Application number: 12/689,312
- Filed: 2010-01-19
- Published: 2010-07-22
- Granted: 2013-10-29
- Earliest claimed: 2009-01-16
- Assignees: Samsung Electronics Co., Ltd.; Industry-University Cooperation Foundation Sogang University
- Applicants: Soon-keun Chang; Rae-hong Park; Ji-won Lee
- Inventors: Soon-keun Chang; Rae-hong Park; Ji-won Lee
- Web page: https://patentgrep.com/patent/US8571312B2

## CPC

- H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values}
- G06T 3/4007: based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403)
- H04N 25/134: {based on three different wavelength filter elements}
- H04N 2209/046: Colour interpolation to calculate the missing colour values
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image interpolating method and apparatus, in which horizontal and vertical Differences of Absolute Inter-channel Differences (DAIDs) are calculated from a color filter array (CFA) image, and an unknown pixel is interpolated in horizontal and vertical directions estimated in consideration of the DAIDs of R, G, and B pixels. Therefore, the image interpolating method and apparatus provide a large Peak Signal and Noise Ratio (PSNR), a Structural Similarity (SSIM), and high visual quality images.

## Claims

### Claim 1

An image interpolating method comprising:
detecting an edge direction by calculating absolute inter-channel differences with respect to left, right, up, and down directions of a central pixel of a color filter array according to pattern characteristics of R, G, and B pixels arranged in the color filter array, the absolute inter-channel differences being calculated by subtracting the absolute value of a difference between a color intensity value of a pixel of a first color channel on one side of the central pixel and a color intensity value of a neighboring pixel of a second color channel on the one side of the central pixel, from the absolute value of a difference between a color intensity value of a pixel of the first color channel on a symmetrically opposite side of the central pixel and a color intensity value of a neighboring pixel of the second color channel on the symmetrically opposite side of the central pixel;
calculating directional interpolation weights for the left, right, up, and down directions, respectively, according to the pattern characteristics of the R, G, and B pixels; and
interpolating an unknown pixel value using color differences between the R, G, and B pixels with respect to the left, right, up, and down directions of the center pixel and the directional interpolation weights according to the detected edge direction.

### Claim 2 (depends on 1)

The image interpolating method of claim 1, wherein the absolute inter-channel differences are differences between values of pixels that are adjacent to each other in the left, right, up, and down directions.

### Claim 3 (depends on 2)

The image interpolating method of claim 2, wherein the detecting of the edge direction comprises calculating a horizontal difference DH for the left and right directions and a vertical difference DV for the up and down directions from the absolute inter-channel differences with respect to the left, right, up, and down directions.

### Claim 4 (depends on 3)

The image interpolating method of claim 3, wherein the detecting of the edge direction comprises, if the central pixel of the color filter array is Ri,j, calculating the horizontal difference DH and the vertical difference DV according to the following Equations 4 and 5:
DH=DHL−DHR=|Ri,j−2−Gi,j−1|−|Ri,j+2|−|Ri,j+2−Gi,j+1| [Equation 4]
DV=DVU−DVD=|Ri−2,j−Gi−1,j|−|Ri+2,j−Gi+1,j| [Equation 5]
where Rp.q and Gp,q denote color intensity values at positions (p,q) in the R and G channels, respectively, and DHL, DHR, DVU, and DVD denote inter-channel differences with respect to the left, right, up, and down directions, respectively.

### Claim 5 (depends on 4)

The image interpolating method of claim 4, wherein the interpolating of the unknown pixel value comprises, if the central pixel of the color filter array is Ri,j, interpolating an unknown G pixel value according to the following Equation 6:
G
i
,
j
′
=
+
{
R
i
,
j
+
γ
H
L
⁢
Δ
⁢
⁢
G
H
L
+
γ
H
R
⁢
Δ
⁢
⁢
G
H
R
γ
H
L
+
γ
H
R
,
D
H
<
D
V
R
i
,
j
+
γ
V
U
⁢
Δ
⁢
⁢
G
V
U
+
γ
V
D
⁢
Δ
⁢
⁢
G
V
D
γ
V
U
+
γ
V
D
,
D
H
>
D
V
R
i
,
j
+
γ
H
L
⁢
Δ
⁢
⁢
G
H
L
+
γ
H
R
⁢
Δ
⁢
⁢
G
H
R
+
γ
V
U
⁢
Δ
⁢
⁢
G
V
U
+
γ
V
D
⁢
Δ
⁢
⁢
G
V
D
γ
H
L
+
γ
H
R
+
γ
V
U
+
γ
V
D
,
otherwise
Equation
⁢
⁢
6
where G′i,j denotes an interpolated G pixel value, ΔGHL, ΔGHR, ΔGVU, and ΔGVD denote color differences defined with respect to the left, right, up, and down directions, respectively, and γHL, γHR, γVU, and γVD denote directional interpolation weights defined with respect to the left, right, up, and down directions, respectively.

### Claim 6 (depends on 3)

The image interpolating method of claim 3, wherein the interpolating of the unknown pixel value comprises, after a G pixel from among unknown pixels is interpolated, interpolating the values of the other unknown pixels using a pixel value G′ of the interpolated G pixel.

### Claim 7 (depends on 6)

The image interpolating method of claim 6, wherein the interpolating of the unknown pixel value comprises, if the central pixel of the color filter array is Bi,j, interpolating an unknown R pixel value according to the following Equation 15:
R
i
,
j
′
=
G
i
,
j
′
+
γ
H
L
⁢
Δ
⁢
⁢
R
H
L
+
γ
H
R
⁢
Δ
⁢
⁢
R
H
R
+
γ
V
U
⁢
Δ
⁢
⁢
R
V
U
+
γ
V
D
⁢
Δ
⁢
⁢
R
V
D
γ
H
L
+
γ
H
R
+
γ
V
U
+
γ
V
D
[
Equation
⁢
⁢
15
]
where R′i,j and G′i,j denotes interpolated pixel values, ΔRHL, ΔRHR, ΔRVU, and ΔRVD denote color differences defined with respect to the left, right, up, and down directions of the center pixel, respectively, ΔRHL=Ri+1,j−1−G′i+1,j−1, ΔRHR=Ri−1,j+1−G′i−1,j+1, ΔRVU=Ri−1,j−1−G′i−1,j−1, and ΔRVD=Ri+1,j+1−G′i+1,j+1, and γHL, γHR, γVU, and γVD denote directional interpolation weights defined with respect to the left, right, up, and down directions, respectively.

### Claim 8 (depends on 7)

The image interpolating method of claim 7, wherein, if the central pixel of the color filter array is Gi,j, the color differences defined with respect to the left, right, up, and down directions in Equation 15 are ΔRHL=R′i,j−1−G′i,j−1, ΔRHR=R′i,j+1−G′i,j+1, ΔRVU=Ri+1,j−G′i−1,j, and ΔRVD=Ri+1,j−G′i+1,j, respectively.

### Claim 9 (depends on 1)

The image interpolating method of claim 1, further comprising adjusting the interpolated pixel value by adding a high frequency component of a known pixel to the interpolated pixel value.

### Claim 10

A non-transitory computer readable recording medium encoded with computer-executable instructions that when executed by a computer perform the following method:
detecting an edge direction by calculating absolute inter-channel differences with respect to left, right, up, and down directions of a central pixel of a color filter array according to pattern characteristics of R, G, and B pixels arranged in the color filter array, the absolute inter-channel differences being calculated by subtracting the absolute value of a difference between a color intensity value of a pixel of a first color channel on one side of the central pixel and a color intensity value of a neighboring pixel of a second color channel on the one side of the central pixel, from the absolute value of a difference between a color intensity value of a pixel of the first color channel on a symmetrically opposite side of the central pixel and a color intensity value of a neighboring pixel of the second color channel on the symmetrically opposite side of the central pixel;
calculating directional interpolation weights for the left, right, up, and down directions, respectively, according to the pattern characteristics of the R, G, and B pixels; and
interpolating an unknown pixel value using color differences between the R, G, and B pixels with respect to the left, right, up, and down directions of the center pixel and the directional interpolation weights according to the detected edge direction.

### Claim 11

An image interpolating apparatus comprising:
an edge direction detection unit configured to detect an edge direction by calculating absolute inter-channel differences with respect to left, right, up, and down directions of a central pixel of a color filter array according to pattern characteristics of R, G, and B pixels arranged in the color filter array, the absolute inter-channel differences being calculated by subtracting the absolute value of a difference between a color intensity value of a pixel of a first color channel on one side of the central pixel and a color intensity value of a neighboring pixel of a second color channel on the one side of the central pixel, from the absolute value of a difference between a color intensity value of a pixel of the first color channel on a symmetrically opposite side of the central pixel and a color intensity value of a neighboring pixel of the second color channel on the symmetrically opposite side of the central pixel;
a weight calculation unit configured to calculate directional interpolation weights for the left, right, up, and down directions, respectively, according to the pattern characteristics of the R, G, and B pixels; and
an interpolation unit configured to interpolate an unknown pixel value using color differences between the R, G, and B pixels with respect to the left, right, up, and down directions and the directional interpolation weights according to the detected edge direction.

### Claim 12 (depends on 11)

The image interpolating apparatus of claim 11, wherein the absolute inter-channel differences are differences between values of pixels that are adjacent to each other in the left, right, up, and down directions.

### Claim 13 (depends on 12)

The image interpolating apparatus of claim 12, wherein the edge direction detection unit calculates a horizontal difference DH for the left and right directions and a vertical difference DV for the up and down directions from the absolute inter-channel differences with respect to the left, right, up, and down directions.

### Claim 14 (depends on 13)

The image interpolating apparatus of claim 13, wherein, if the central pixel of the color filter array is Ri,j the edge direction detection unit calculates the horizontal difference DH and the vertical difference DV according to the following Equations 4 and 5:
DH=DHL−DHR−|Ri,j−2−Gi,j−1|−|Ri,j+2−Gi,j+1| [Equation 4]
DV=DVU−DVD=|Ri−2,j−Gi−1,j|−|Ri+2,j−Gi+1,j| [Equation 5]
where Rp.q and Gp,q denote color intensity values at positions (p,q) in the R and G channels, respectively, and ΔDHL, ΔDHR, ΔDVU, and DVD denote inter-channel differences with respect to the left, right, up, and down directions of the center pixel, respectively.

### Claim 15 (depends on 14)

The image interpolating apparatus of claim 14, wherein, if the central pixel of the color filter array is Ri,j, the interpolation unit interpolates an unknown G pixel value according to the following Equation 6:
G
i
,
j
′
=
+
{
R
i
,
j
+
γ
H
L
⁢
Δ
⁢
⁢
G
H
L
+
γ
H
R
⁢
Δ
⁢
⁢
G
H
R
γ
H
L
+
γ
H
R
,
D
H
<
D
V
R
i
,
j
+
γ
V
U
⁢
Δ
⁢
⁢
G
V
U
+
γ
V
D
⁢
Δ
⁢
⁢
G
V
D
γ
V
U
+
γ
V
D
,
D
H
>
D
V
R
i
,
j
+
γ
H
L
⁢
Δ
⁢
⁢
G
H
L
+
γ
H
R
⁢
Δ
⁢
⁢
G
H
R
+
γ
V
U
⁢
Δ
⁢
⁢
G
V
U
+
γ
V
D
⁢
Δ
⁢
⁢
G
V
D
γ
H
L
+
γ
H
R
+
γ
V
U
+
γ
V
D
,
otherwise
Equation
⁢
⁢
6
where G′i,j denotes an interpolated G pixel value, ΔGHL, ΔGHR, ΔGVU, and ΔGVD denote color differences defined with respect to the left, right, up, and down directions, respectively, and γHL, γHR, γVU, and γVD denote directional interpolation weights defined with respect to the left, right, up, and down directions, respectively.

### Claim 16 (depends on 13)

The image interpolating apparatus of claim 13, wherein, after a G pixel from among unknown pixels is interpolated, the interpolation unit interpolates the values of the other unknown pixels using a pixel value G′ of the interpolated G pixel.

### Claim 17 (depends on 16)

The image interpolating apparatus of claim 16, wherein, if the central pixel of the color filter array is Bi,j, the interpolation unit interpolates an unknown R pixel value according to the following Equation 15:
R
i
,
j
′
=
G
i
,
j
′
+
γ
H
L
⁢
Δ
⁢
⁢
R
H
L
+
γ
H
R
⁢
Δ
⁢
⁢
R
H
R
+
γ
V
U
⁢
Δ
⁢
⁢
R
V
U
+
γ
V
D
⁢
Δ
⁢
⁢
R
V
D
γ
H
L
+
γ
H
R
+
γ
V
U
+
γ
V
D
[
Equation
⁢
⁢
15
]
where R′i,j and G′i,j denotes interpolated pixel values, ΔRHL, ΔRHR, ΔRVU, and ΔRVD denote color differences defined with respect to the left, right, up, and down directions of the center pixel, respectively, ΔRHL=Ri+1,j−1−G′i+1,j−1, ΔRHR=Ri−1,j+1−G′i−1,j+1, ΔRVU=Ri−1,j−1−G′i−1,j−1, and γHL, γHR, γVU, and γVD denote directional interpolation weights defined with respect to the left, right, up, and down directions, respectively.

### Claim 18 (depends on 17)

The image interpolating apparatus of claim 17, wherein, if the central pixel of the color filter array is Gi,j, the color differences defined with respect to the left, right, up, and down directions in Equation 15 are ΔRHL=R′i,j−1−G′i,j−1, ΔRHR=R′i,j+1−G′i,j+1, ΔRVU−Ri−1,j−G′i−1,j, and ΔRVD=Ri+1,j−G′i+1,j, respectively.

### Claim 19 (depends on 11)

The image interpolating apparatus of claim 11, further comprising an adjusting unit for adjusting the interpolated pixel value by adding a high frequency component of a known pixel to the interpolated pixel value.

### Claim 20

An image interpolating method comprising:
detecting an edge direction by calculating absolute inter-channel differences with respect to left, right, up, and down directions of a central pixel of a color filter array according to pattern characteristics of different color pixels arranged in the color filter array, the absolute inter-channel differences being calculated by subtracting the absolute value of a difference between a color intensity value of a pixel of a first color channel on one side of the central pixel and a color intensity value of a neighboring pixel of a second color channel on the one side of the central pixel, from the absolute value of a difference between a color intensity value of a pixel of the first color channel on a symmetrically opposite side of the central pixel and a color intensity value of a neighboring pixel of the second color channel on the symmetrically opposite side of the central pixel;
calculating directional interpolation weights for the left, right, up, and down directions, respectively, according to the pattern characteristics of the different color pixels; and
interpolating an unknown pixel value using color differences between the different color pixels with respect to the left, right, up, and down directions of the center pixel and the directional interpolation weights according to the detected edge direction.