# US 2014/0193041 A1 · System and method for resolution enhancement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0193041 A1
- Document ID: US20140193041A1
- Status: Published application
- Kind: A1
- Application number: 14/238,735
- Filed: 2011-12-12
- Published: 2014-07-10
- Earliest claimed: 2011-12-12
- Assignees: Aselsan Elektronik Sanayi Ve Ticaret Anonim Sirketi
- Inventors: Lutfi Murat Gevrekci
- Web page: https://patentgrep.com/patent/US20140193041A1

## CPC

- G06T 3/4076: using the original low-resolution images to iteratively correct the high-resolution images
- G06T 5/70: Denoising; Smoothing
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present invention relates to a system (1) and method (100) for resolution enhancement which enables the depth resolution to be enhanced without using gimbal, performs depth description independent from lighting conditions and camera parameters in high resolution.

## Claims

### Claim 1

A resolution enhancement method (100) enabling the images acquired in low resolution to become high resolution and characterized by the steps of
initializing the gradient buffer (7) and the image buffer (101),
acquiring low resolution image from the imaging unit (2) using data received from the exposure generation module (31) and offset generation module (32) (102),
recording the image received to the database (4) (103),
implementing resolution enhancement process (104),
displaying the final image to the user (105).

### Claim 2 (depends on 1)

A resolution enhancement method (100) according to claim 1, characterized in that the step of implementing resolution enhancement process (104) comprises the sub-steps of
determining the motion quantity of the low resolution image received last from the imaging unit (2) with respect to the previous low resolution image (120),
aligning the last low resolution image with the previous low resolution image using the estimated motion quantity in 120th step (121),
aligning the last image on the image buffer (8) with the previous image by using the estimated motion in 120th step (122),
aligning the last image on the gradient buffer (7) with the previous image by using the estimated motion in 120th step (123),
determining a weight function by using saturated and noisy data in the aligned image acquired in step 123 (124),
finding the difference between the images in the image buffer (8) and computing the gradient buffer (7) and in (125),
averaging the differences found in step 125 in certain portions to the image aligned in step 123 (126),
storing the image acquired in step 126 in gradient buffer (7) (127),
combining the image acquired in step 121 and the image in the gradient buffer (7) in certain content ratio (128),
suppressing the high frequency elements in the acquired image (129),
registering the image acquired in step 129 to image buffer (8) (130).

### Claim 3 (depends on 2)

A resolution enhancement method (100) according to claim 2, characterized in that the gradient buffer is updated according to xm+1=xm+αĪm+1 Formula in step of storing the image acquired in step 126 in the gradient buffer (7) (in other words step 127).

### Claim 4 (depends on 2, 3)

A resolution enhancement method (100) according to any one of the claim 2 or 3, characterized in that the suppressing and photometric normalization process is realized according to
q
^
N

(
n
1
,
n
2
)
=
f
-
1

(
q
^

(
n
1
,
n
2
)
)
α
i
in step of supressing high frequency elements in the acquired image (in other word step in 129).

### Claim 5 (depends on 2, 3, 4)

A resolution enhancement method (100) according to any one of the claims 2 to 4, characterized in that the step of finding the difference between the images in the gradient buffer (7) and in the image buffer (8) (125) comprises the sub-steps of
acquiring low resolution image from the imaging unit (2) (150),
tonal adjustment of the image in the image buffer (8) with respect to the image acquired in step 150 (151),
downscaling the image in the imaging buffer (8) to the size of the image acquired in step 150 (152),
determining the depth difference between the image acquired in step 152 and the image acquired in step 150 (153),
upscaling the difference determined in step 153 to the size of the image in the image buffer (8) again (154),
multiplying the image acquired in step 154 with the weight function acquired in step 124 (155).

### Claim 6

A resolution enhancement method (100) according to any one of the preceding claims, characterized in that the step of taking low resolution image from the imaging unit (2) by using the data received from the exposure generation module (31) and the offset generation module (32) (in other words step 102) is repeated by using different exposure and offset information generated by exposure generation module (31) and the offset generation module (32).

### Claim 7 (depends on 2, 3, 4, 5, 6)

A resolution enhancement method (100) according to any one of the claims 2 to 6, characterized in the step of determining the motion quantity of the low resolution image last received from the imaging unit (2) with respect to the previous low resolution image (in other words step in 120), the image is subjected to contrast stretching process according to a pixel intensity set.

### Claim 8 (depends on 7)

A resolution enhancement method (100) according to claim 7, characterized in that the contrast stretching process is performed according to
f
c

(
l

(
x
,
y
)
)
=
1
1
+

-
γ

(
l

(
x
,
y
)
-
c
)
.

### Claim 9 (depends on 2, 3, 4, 5, 6, 7, 8)

A resolution enhancement method (100) according to any one of the claims 2 to 8, characterized in corner detection algorithm is applied to the acquired image in order to produce a three-dimensional corner response map invariant to photometric variations.

### Claim 10 (depends on 9)

A resolution enhancement method (100) according to claim 9, characterized in that the similarity of the corners is computed according to
ρ
A
,
B
=
∑
i
=
1
n


(
A
i
-
A
_
)

(
B
i
-
B
_
)
∑
i
=
1
n


(
A
i
-
A
_
)
2

∑
i
=
1
n


(
B
i
-
B
_
)
2

### Claim 11 (depends on 5, 6, 7, 8, 9, 10)

A resolution enhancement method (100) according to any one of the claims 5 to 10, characterized in step of determining the depth difference between the image acquired in step 152 and the image acquired in step 150 (in other words in step 153) the difference between images is determined computing the mean depth information of the images and performing residual computation according to the said difference computed.

### Claim 12 (depends on 11)

A resolution enhancement method (100) according to claim 11, characterized in that the mean depth computation is performed according to
FP
i

(
l
1
,
l
2
)
=
∑
n
1
,
n
2


h
i

(
l
1
,
l
2
;
n
1
,
n
2
)

q
^
N

(
n
1
,
n
2
)
Formula.

### Claim 13 (depends on 11, 12)

A resolution enhancement method (100) according to any one of the claim 11 or 12, characterized in that the residual computation is performed according to ri(l1,l2)=Di(l1,l2)−FPi(l1,l2) Formula.

### Claim 14

A resolution enhancement system (1) comprising
at least one imaging unit (2) which enables to acquire image,
at least one data transfer unit (3) which has at least one exposure generation module (31) enabling different exposure times to be generated for the images to be taken, at least one offset generation module (32) enabling the data about the offset of the imaging unit (2) to be generated, and at least one data transfer module (33) enabling the data to be received and transferred,
at least one database (4) which enables the images acquired to be stored,
at least one motion unit (5) which enables the movement of the imaging unit (2) in tree dimension according to the data acquired by the offset generation module (32),
at least control unit (6) which enables the images acquired to be processed, and characterized by
at least one gradient buffer (7) wherein the differences between the images are stored.

### Claim 15 (depends on 14)

A resolution enhancement system (1) according to claim 14, characterized by at least one image buffer (8) wherein the high resolution depth image is stored.

### Claim 16 (depends on 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)

A resolution enhancement system (1) according to any one of the claim 14 or 15, characterized control unit (6) operating according to the method in any one of the claims 1 to 13.