# US 2015/0104097 A1 · Image processing apparatus and image processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0104097 A1
- Document ID: US20150104097A1
- Status: Published application
- Kind: A1
- Application number: 14/574,698
- Filed: 2014-12-18
- Published: 2015-04-16
- Earliest claimed: 2012-06-27
- Assignees: Olympus Corporation
- Applicants: Olympus Corporation
- Inventors: Toshihiro TSUZUKI
- Web page: https://patentgrep.com/patent/US20150104097A1

## CPC

- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/85: {Stereo camera calibration}
- G06T 2207/10012: Stereo images
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image processing apparatus includes an image acquisition unit acquiring a plurality of images, a corresponding point acquisition unit, a first fundamental matrix estimation unit, an epipole coordinate deriving unit, an epipole coordinate determination unit, and a fundamental matrix determination unit. The corresponding point acquisition unit acquires first corresponding points. The first fundamental matrix estimation unit calculates first fundamental matrices based on the first corresponding points. The epipole coordinate deriving unit calculates first epipole coordinates that correspond to the first fundamental matrices. The epipole coordinate determination unit determines one of the first epipole coordinates as a second epipole coordinate. The fundamental matrix determination unit determines the first fundamental matrix corresponding to the second epipole coordinate as a second fundamental matrix.

## Claims

### Claim 1

An image processing apparatus comprising:
an image acquisition unit configured to acquire a plurality of images;
a corresponding point acquisition unit configured to acquire a plurality of first corresponding points that correspond to each other between the images;
a first fundamental matrix estimation unit configured to calculate a plurality of first fundamental matrices based on the first corresponding points;
an epipole coordinate deriving unit configured to calculate, for at least one image, a plurality of first epipole coordinates that respectively correspond to the first fundamental matrices;
an epipole coordinate determination unit configured to determine one of the first epipole coordinates as a second epipole coordinate by using a predetermined statistical technique; and
a fundamental matrix determination unit configured to determine the first fundamental matrix corresponding to the second epipole coordinate as a second fundamental matrix.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, wherein the epipole coordinate determination unit comprises:
a median calculation unit configured to calculate a median of the first epipole coordinates;
a distance calculation unit configured to calculate distances between each of the first epipole coordinates and the median; and
an epipole selection unit configured to determine the first epipole coordinate having the smallest distance as the second epipole coordinate among the first epipole coordinates.

### Claim 3 (depends on 1)

The image processing apparatus according to claim 1, wherein the epipole coordinate determination unit comprises:
a histogram creation unit configured to count a frequency of the first epipole coordinates;
a barycentric position calculation unit configured to determine a barycentric position of a region having a high frequency;
a distance calculation unit configured to calculate distances between each of the first epipole coordinates and the barycentric position; and
an epipole selection unit configured to determine one of the first epipole coordinates having the smallest distance as the second epipole coordinate among the first epipole coordinates.

### Claim 4 (depends on 1)

The image processing apparatus according to claim 1, further comprising:
a motion estimation unit configured to estimate a difference in position and attitude between cameras which have taken the images; and
a three-dimensional coordinate estimation unit configured to estimate three-dimensional coordinates of a subject included in the images based on the difference in position and attitude.

### Claim 5 (depends on 1)

The image processing apparatus according to claim 1, wherein a number of the first fundamental matrices is determined based on at least one of imaging device information, scene determination information, motion vector information, and epipole coordinate distribution information.

### Claim 6

An imaging device comprising:
an imaging unit configured to form images;
an image acquisition unit configured to acquire the images formed at different time;
a corresponding point acquisition unit configured to acquire a plurality of first corresponding points that correspond to each other between the images;
a first fundamental matrix estimation unit configured to calculate a plurality of first fundamental matrices based on the first corresponding points;
an epipole coordinate deriving unit configured to calculate, for at least one image, a plurality of first epipole coordinates that respectively correspond to the first fundamental matrices;
an epipole coordinate determination unit configured to determine one of the first epipole coordinates as a second epipole coordinate by using a predetermined statistical technique;
a fundamental matrix determination unit configured to determine the first fundamental matrix corresponding to the second epipole coordinate as a second fundamental matrix; and
a correction unit configured to correct a shake between the images based on the second fundamental matrix.

### Claim 7

An image processing method comprising:
acquiring a plurality of images;
acquiring a plurality of first corresponding points that correspond to each other between the images;
calculating a plurality of first fundamental matrices based on the first corresponding points;
calculating, for at least one image, a plurality of first epipole coordinates that respectively correspond to the first fundamental matrices;
determining one of the first epipole coordinates as a second epipole coordinate by using a predetermined statistical technique; and
determining the first fundamental matrix corresponding to the second epipole coordinate as a second fundamental matrix.

### Claim 8

A non-transitory computer-readable storage medium recording a program which causes a computer to:
acquire a plurality of images;
acquire a plurality of first corresponding points that correspond to each other between the images;
calculate a plurality of first fundamental matrices based on the first corresponding points;
calculate, for at least one image, a plurality of first epipole coordinates that respectively correspond to the first fundamental matrices;
determine one of the first epipole coordinates as a second epipole coordinate by using a predetermined statistical technique; and
determine the first fundamental matrix corresponding to the second epipole coordinate as a second fundamental matrix.
