# US 2014/0152838 A1 · Method and apparatus for processing image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0152838 A1
- Document ID: US20140152838A1
- Status: Published application
- Kind: A1
- Application number: 13/919,154
- Filed: 2013-06-17
- Published: 2014-06-05
- Earliest claimed: 2012-12-05
- Assignees: Industry-Academic Cooperation Foundation, Yonsei University; Samsung Techwin Co., Ltd.
- Applicants: Industry-Academic Cooperation Foundation, Yonsei University; Samsung Techwin Co., Ltd.
- Inventors: Jeong-Ahn LEE; Min-seok LEE; Ji-yong KWON; Moon-gi KANG
- Web page: https://patentgrep.com/patent/US20140152838A1

## CPC

- H04N 25/131: including elements passing infrared wavelengths
- H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values}
- H04N 25/135: {based on four or more different wavelength filter elements}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Exemplary embodiments disclose a method of processing an image. The method includes: offsetting aliasing by analyzing frequency characteristics of at least one color channel and a NIR channel, of a MSFA pattern image; generating a high resolution base image; and offsetting an artifact from the high resolution base image by weighting the high resolution base image and a blur image, based on a pixel difference value between the blur image and the high resolution base image.

## Claims

### Claim 1

A method of processing an image, the method comprising:
offsetting aliasing by analyzing frequency characteristics of at least one color channel and a near-infrared (NIR) channel, of a multi-spectral filter array (MSFA) pattern image;
generating a high resolution base image by amplifying resolutions of the at least one color channel and the NIR channel, from which the aliasing is removed, up to a resolution of a base image; and
offsetting an artifact from the high resolution base image by weighting and combining the high resolution base image and a blur image obtained by blurring the high resolution base image, based on a pixel difference value between the blur image and the high resolution base image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the offsetting of the aliasing comprises combining the at least one color channel and the NIR channel of the MSFA pattern image, while considering different phase characteristics shown in the at least one color channel and the NIR channel.

### Claim 3 (depends on 1)

The method of claim 1, wherein the generating of the high resolution base image comprises amplifying the resolutions of the at least one color channel and the NIR channel, from which the aliasing is removed, up to the resolution of the base image using a sinc function.

### Claim 4 (depends on 3)

The method of claim 3, wherein the sinc function uses a cutoff frequency wc equal to or above 0.5Π.

### Claim 5 (depends on 3)

The method of claim 3, wherein the cutoff frequency wc of the sinc function is adaptively set according to statistical characteristics of a location region obtained from the MSFA pattern image, and
when a local variance is high, the cutoff frequency wc is set to increase a pass-band of the sinc function, and
when the local variance is low, the cutoff frequency wc is set to decrease the pass-band of the sinc function.

### Claim 6 (depends on 3)

The method of claim 3, wherein Base(i,j)=(Gs(i,j)+Ns(i,j)+Rs(i,j)+Bs(i,j))*Sinc(i,j),
wherein Rs(i,j), Gs(i,j), and Bs(i,j) denote the at least one color channel of the MSFA pattern image, Ns(i,j) denotes the NIR channel of the MSFA pattern image,
the aliasing is removed via a sum of the at least one color channel and the NIR channel, and the resolutions of the at least one color channel and the NIR channel are amplified up to the resolution of the base image via the sinc function.

### Claim 7 (depends on 3)

The method of claim 3, wherein the cutoff frequency wc of the sinc function is set based on a tan h function using a local variance of the MSFA pattern image as a variable.

### Claim 8 (depends on 7)

The method of claim 7, wherein the cutoff frequency wc of the sinc function is adaptively set according to:
ω
c
adap
=
ω
c
high
-
ω
c
low
max

(
σ
local
2
)
-
min

(
σ
local
2
)
×
(
x
)
+
ω
c
low
,
wherein x is set according to:
x
=
max

(
σ
local
2
)
2
×
tanh

(
p
max

(
σ
local
2
)

σ
local
2
-
k
)
,
wherein x is an input for obtaining the cutoff frequency wc, and denotes a variance of the local region adjusted according to an s-curve (tan h) weight, and p and k are factors for adjusting the local region variance.

### Claim 9 (depends on 1)

The method of claim 1, wherein the offsetting of the artifact comprises offsetting the artifact by weighting and combining the blur image and the high resolution base image,
wherein,
when many artifacts are observed, weighting the blur image more,
when few artifacts are observed, weighting the high resolution base image more, and
a sum of weights of the blur image and the high resolution base image is 1.

### Claim 10

A method of processing an image, the method comprising:
offsetting aliasing by analyzing frequency characteristics of at least one color channel and a near-infrared (NIR) channel, of a multi-spectral filter array (MSFA) pattern image, and combining the at least one color channel and the NIR channel such that phase characteristics of the at least one color channel and the NIR channel offset each other;
generating a high resolution base image by amplifying resolutions of the at least one color channel and the NIR channel, from which the aliasing is removed, up to a resolution of a base image using a sinc function, wherein a cutoff frequency wc of the sinc function is adaptively set according to statistical characteristics of a local region obtained from the MSFA pattern image; and
offsetting an artifact of the high resolution base image by weighting and combining the high resolution base image and a blur image obtained by blurring the high resolution base image, based on a pixel difference value between the blur image and the high resolution base image.

### Claim 11 (depends on 10)

The method of claim 10, wherein Base(i,j)=(Gs(i,j)+Ns(i,j)+Rs(i,j)+Bs(i,j))*Sinc(i,j),
wherein Rs(i,j), Gs(i,j), and Bs(i,j) denote the at least one color channel of the MSFA pattern image, Ns(i,j) denotes the NIR channel of the MSFA pattern image, the aliasing is removed via a sum of the at least one color channel and the NIR channel, and the resolutions of the at least one color channel and the NIR channel are amplified up to the resolution of the base image via the sinc function.

### Claim 12 (depends on 10)

The method of claim 10, wherein the cutoff frequency wc of the sinc function is adaptively set according to statistical characteristics of a location region obtained from the MSFA pattern image, and
when a local variance is high, the cutoff frequency wc is set to increase a pass-band of the sinc function, and
when the local variance is low, the cutoff frequency wc is set to decrease the pass-band of the sinc function.

### Claim 13 (depends on 10)

The method of claim 10, wherein the cutoff frequency wc of the sinc function is adaptively set according to:
ω
c
adap
=
ω
c
high
-
ω
c
low
max

(
σ
local
2
)
-
min

(
σ
local
2
)
×
(
x
)
+
ω
c
low
,
wherein x is set according to:
x
=
max

(
σ
local
2
)
2
×
tanh

(
p
max

(
σ
local
2
)

σ
local
2
-
k
)
,
wherein x is an input for obtaining the cutoff frequency wc and denotes a variance of the local region adjusted according to an s-curve (tan h) weight, and p and k are factors for adjusting the local region variance.

### Claim 14 (depends on 10)

The method of claim 10, wherein the offsetting of the artifact comprises offsetting the artifact by weighting and combining the blur image and the high resolution base image,
wherein,
when many artifacts are observed, weighting the blur image more,
when few artifacts are observed, weighting the high resolution base image more, and
a sum of weights of the blur image and the high resolution base image is 1.

### Claim 15

An apparatus for processing an image, the apparatus comprising:
an aliasing offset device configured to offset aliasing by analyzing frequency characteristics of at least one color channel and a near-infrared (NIR) channel, of a multi-spectral filter array (MSFA) pattern image;
a high resolution base image generator configured to generate a high resolution base image by amplifying resolutions of the at least one color channel and the NIR channel from which the aliasing is removed, up to a resolution of a base image; and
an artifact offset device configured to offset an artifact from the high resolution base image by weighting and combining the high resolution base image and a blur image obtained by blurring the high resolution base image, based on a pixel difference value between the blur image and the high resolution base image.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the aliasing offset device combines the at least one color channel and the NIR channel of the MSFA pattern image, while considering different phase characteristics shown in the at least one color channel and the NIR channel.

### Claim 17 (depends on 15)

The apparatus of claim 15, wherein the high resolution base image generator amplifies the resolutions of the at least one color channel and the NIR channel, from which the aliasing is removed, up to the resolution of the base image using a sinc function.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein a cutoff frequency we of the sinc function is adaptively set according to statistical characteristics of a location region obtained from the MSFA pattern image, and
when a local variance is high, the cutoff frequency wc is set to increase a pass-band of the sinc function, and
when the local variance is low, the cutoff frequency wc is set to decrease the pass-band of the sinc function.

### Claim 19 (depends on 15)

The apparatus of claim 15, wherein the artifact offset unit offsets the artifact by weighting and combining the blur image and the high resolution base image,
wherein,
when many artifacts are observed, weighting the blur image more,
when few artifacts are observed, weighting the high resolution base image more, and
a sum of weights of the blur image and the high resolution base image is 1.

### Claim 20

An apparatus for processing an image, the apparatus comprising:
an aliasing offset device configured to offset aliasing by analyzing frequency characteristics of at least one color channel and a near-infrared (NIR) channel, of a multi-spectral filter array (MSFA) pattern image, and combining the at least one color channel and the NIR channel such that phase characteristics of the at least one color channel and the NIR channel offset each other;
a high resolution base image generator configured to generate a high resolution base image by amplifying resolutions of the at least one color channel and the NIR channel, from which the aliasing is removed, up to a resolution of a base image using a sinc function, wherein a cutoff frequency wc of the sinc function is adaptively set according to statistical characteristics of a local region obtained from the MSFA pattern image; and
an artifact offset device configured to offset an artifact of the high resolution base image by weighting and combining the high resolution base image and a blur image obtained by blurring the high resolution base image, based on a pixel difference value between the blur image and the high resolution base image.