# US 8,279,264 B2 · Method for photographic panoramic image when threshold exceeds comparison between current and previous images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,279,264 B2
- Document ID: US8279264B2
- Status: Granted
- Kind: B2
- Application number: 12/082,103
- Filed: 2008-04-07
- Published: 2008-10-16
- Granted: 2012-10-02
- Earliest claimed: 2007-04-12
- Assignees: Samsung Electronics Co., Ltd.; Seoul National University Industry Foundation
- Inventors: Jae-Won Moon; Soo-Kyun Kim; Sung-Dae Cho; Yun-Je Oh; Hee-Won Jung; Nam-Ik Cho; Sang-Hwa Lee; Seong-Jong Ha
- Web page: https://patentgrep.com/patent/US8279264B2

## CPC

- G03B 37/04: with cameras or projectors providing touching or overlapping fields of view
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- H04N 5/76: Television signal recording
- H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters}
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Disclosed is a method for photographing a panoramic image including the steps of recognizing movement of a corresponding photographing apparatus by comparing a current real-time input image with a previous image through a motion estimation mechanism with exposure compensation, determining a time to photograph each next picture by determining whether movement in a photography direction reaches a preset threshold value, and photographing each next picture by manual or automatic operation at the determined time.

## Claims

### Claim 1

A method for photographing a panoramic image composed of a plurality of pictures, the method comprising the steps of:
comparing a current real-time input image with a previous image through a motion estimation mechanism with exposure compensation to recognize movement of a corresponding photographing apparatus;
determining a time to photograph each next picture by determining whether the recognized movement in a photography direction reaches a preset threshold value; and
photographing each next picture by manual or automatic operation at the determined time;
wherein in the motion estimation mechanism with the exposure compensation comprises the steps of:
compensating pixels of each image block within compared frames in accordance with an average value of the image block, and determining a motion vector between matching image blocks using error information of respective compared pixels between image blocks within compared frames.

### Claim 2 (depends on 1)

The method as claimed in claim 1, wherein the error information comprises SAD (Sum of Absolute Difference) or SSD (Sum of Squared Difference), which is obtained by
modified SAD
E
′
⁡
(
m
,
n
)
=
∑
x
,
y
∈
block
⁢

I
1
⁡
(
x
,
y
)
-
I
_
1
-
(
I
2
⁡
(
x
+
m
,
y
+
n
)
-
I
_
2
⁡
(
m
,
n
)
)

modified SSD
E
⁡
(
m
,
n
)
=
∑
x
,
y
∈
block
⁢
{
I
1
⁡
(
x
,
y
)
-
I
_
1
-
(
I
2
⁡
(
x
+
m
,
y
+
n
)
-
I
_
2
⁡
(
m
,
n
)
)
}
2
,
wherein I1 and I2 indicate pixel values of positions corresponding to x, y and x+m, y+n on the image blocks within a previous frame and a current frame, respectively, and
Ī1 and Ī2 indicate average values of pixels corresponding to blocks within the previous frame and the current frame, respectively.

### Claim 3 (depends on 2)

The method as claimed in claim 2, wherein in obtaining the average values of the pixels, an integral image is prepared with regard to the current frame, and then, an average value of pixels with regard to each image block is obtained using the integral image.

### Claim 4 (depends on 2)

The method as claimed in claim 2, wherein the pixel values comprise luminance or color.

### Claim 5 (depends on 1)

The method as claimed in claim 1, wherein each of the image blocks is variably set according to information selected from the group consisting of: preview performance, a distance from a subject, and a photography mode.

### Claim 6 (depends on 5)

The method as claimed in claim 5, wherein in addition to the image blocks, a search range for searching a matching block is also variably set.

### Claim 7 (depends on 1)

The method as claimed in claim 1, wherein in determining the motion vector between the image blocks, a single-block search mechanism is used.

### Claim 8 (depends on 1)

The method as claimed in claim 1, wherein when the motion estimation mechanism is performed on the frames, the frames are projected on a preset panoramic projection space.

### Claim 9 (depends on 1)

The method as claimed in claim 1, wherein in comparing the frames, some frames are skipped according to a preset value.

### Claim 10

A method for photographing a panoramic image, in which movement of a photographing apparatus is recognized, the method comprising the steps of:
determining matching image blocks between a current real-time input frame and a previous frame; and
determining movement of the photographing apparatus by determining a motion vector between the determined matching image blocks,
wherein in determining the matching image blocks, pixels of each of the image blocks within respective frames are compensated in accordance with an average value of a corresponding image block, and the motion vector between the matching image blocks is determined by using error information on the compared compensated pixels between the image blocks within the respective frames.

### Claim 11 (depends on 10)

The method as claimed in claim 10, wherein the error information comprises SAD (Sum of Absolute Difference) or SSD (Sum of Squared Difference), which is obtained by modified SAD
E
′
⁡
(
m
,
n
)
=
∑
x
,
y
∈
block
⁢

I
1
⁡
(
x
,
y
)
-
I
_
1
-
(
I
2
⁡
(
x
+
m
,
y
+
n
)
-
I
_
2
⁡
(
m
,
n
)
)

modified SSD
E
⁡
(
m
,
n
)
=
∑
x
,
y
∈
block
⁢
{
I
1
⁡
(
x
,
y
)
-
I
_
1
-
(
I
2
⁡
(
x
+
m
,
y
+
n
)
-
I
_
2
⁡
(
m
,
n
)
)
}
2
,
wherein I1 and I2 indicate pixel values of positions corresponding to x, y and x+m, y+n on the image blocks within a previous frame and a current frame, respectively and
Ī1 and Ī2 indicate average values of pixels corresponding to blocks within the previous frame and the current frame, respectively.

### Claim 12 (depends on 10)

The method as claimed in claim 10, wherein in obtaining the average values of the pixels, an integral image is prepared with regard to the current frame, and an average value of pixels with regard to each image block is obtained by using the integral image.

### Claim 13 (depends on 10)

The method as claimed in claim 10, wherein the pixel values comprise luminance or color.

### Claim 14 (depends on 10)

The method as claimed in claim 10, wherein in determining the motion vector between the image blocks, a single-block search mechanism is used.

### Claim 15 (depends on 10)

The method as claimed in claim 10, wherein each of the image blocks and a search range for searching a matching block are both variably set according to information selected from the group consisting of: preview performance, a distance from a subject, and a photography mode.

### Claim 16 (depends on 10)

The method as claimed in claim 10, wherein when a motion estimation mechanism is performed on the frames, the frames are previously projected on a preset panoramic projection space.

### Claim 17 (depends on 10)

The method as claimed in claim 10, wherein in comparing the frames, some frames are skipped according to a preset value.

### Claim 18

An apparatus for photographing a panoramic image composed of a plurality of pictures, the apparatus comprising:
a processor in communication with a memory, the memory containing code which when accessed by said processor causes said processor to execute the steps of:
comparing a current real-time input image with a previous image through a motion estimation mechanism with exposure compensation to recognize movement of a corresponding photographing apparatus;
determining a time to photograph each next picture by determining whether the recognized movement in a photography direction reaches a preset threshold value; and
photographing each next picture by manual or automatic operation at the determined time;
wherein in the motion estimation mechanism with the exposure compensation compensates pixels of each image block within compared frames in accordance with an average value of the image block, and determines a motion vector between matching image blocks using error information of respective compared pixels between image blocks within corn eared frames.
