# US 10,484,665 B2 · Camera parameter set calculation method, recording medium, and camera parameter set calculation apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,484,665 B2
- Document ID: US10484665B2
- Status: Granted
- Kind: B2
- Application number: 15/952,298
- Filed: 2018-04-13
- Published: 2018-10-25
- Granted: 2019-11-19
- Earliest claimed: 2017-04-20
- Assignees: Panasonic Intellectual Property Management Co., Ltd.
- Applicants: Panasonic Intellectual Property Management Co., Ltd.
- Inventors: Kunio Nobori; Nobuhiko Wakai; Satoshi Sato; Takeo Azuma
- Web page: https://patentgrep.com/patent/US10484665B2

## CPC

- H04N 13/246: Calibration of cameras
- G06T 7/85: {Stereo camera calibration}
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- G06T 2207/10012: Stereo images
- G06T 2207/10024: Color image
- G06T 2207/30208: Marker matrix
- H04N 17/002: {for television cameras}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A camera parameter set calculation method includes acquiring a first image from a first camera, a second image from a second camera, first and second camera parameter sets of the first camera and the second camera, calculating three-dimensional coordinate sets on the basis of the first image, the second image, the first camera parameter set, and the second camera parameter set, determining first pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the first image and second pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the second image, calculating an evaluation value on the basis of the pixel values at the first pixel coordinate pairs and the pixel values at the second pixel coordinate pairs, and updating the first camera parameter set and the second camera parameter set on a basis of the evaluation value.

## Claims

### Claim 1

A camera parameter set calculation method comprising:
(a1) acquiring a first image captured by a first camera and a second image captured by a second camera;
(a2) acquiring a first camera parameter set including one or more camera parameters of the first camera and a second camera parameter set including one or more camera parameters of the second camera;
(a3) calculating three-dimensional coordinate sets on an object on a basis of the first image, the second image, the first camera parameter set, and the second camera parameter set, the first image including an image of the object, the second image including an image of the object;
(a4) determining first pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the first image on a basis of the first camera parameter set and determining second pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the second image on a basis of the second camera parameter set;
(a5) calculating an evaluation value on a basis of pixel values at the first pixel coordinate pairs in the first image and pixel values at the second pixel coordinate pairs in the second image;
(a6) updating the first camera parameter set and the second camera parameter set on a basis of the evaluation value; and
(a7) outputting the updated first camera parameter set and second camera parameter set,
wherein at least one of processes (a1) to (a7) is performed by a processor,
the three-dimensional coordinate sets are represented by (x1, y1, z1) . . . (xi, yi, zi) . . . (xn, yn, zn), the first pixel coordinate pairs are represented by (u11, v11) . . . (u1i, v1i) . . . (u1n, v1n), and the second pixel coordinate pairs are represented by (u21, v21) . . . (u2i, v2i) . . . (u2n, v2n), where 1<i≤n, and i and n are natural numbers,
the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u1i, v1i), the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u2i, v2i), the pixel value at the pixel coordinate pair (u1i, v1i) is represented by I1(u1i, v1i), and the pixel value at the pixel coordinate pair (u2i, v2i) is represented by I2(u2i, v2i), and
in process (a5), the evaluation value is calculated on a basis of a difference Di between the pixel values I1(u1i, v1i) and I2(u2i, v2i).

### Claim 2 (depends on 1)

The camera parameter set calculation method according to claim 1, wherein in process (a4), the first pixel coordinate pairs are determined by coordinate-transforming each of the three-dimensional coordinate sets using one or more camera parameters included in the first camera parameter set, and the second pixel coordinate pairs are determined by coordinate-transforming each of the three-dimensional coordinate sets using one or more camera parameters included in the second camera parameter set.

### Claim 3 (depends on 1)

The camera parameter set calculation method according to claim 1, wherein in process (a5), the evaluation value is calculated on a basis of Σi=1nDi.

### Claim 4 (depends on 1)

The camera parameter set calculation method according to claim 1, wherein in process (a5), the evaluation value is calculated on a basis of Σi=1n|Di|.

### Claim 5 (depends on 1)

The camera parameter set calculation method according to claim 1, wherein in process (a5), the evaluation value is calculated on a basis of Σi=1n(Di×Di).

### Claim 6 (depends on 1)

The camera parameter set calculation method according to claim 1, wherein (b1) information about detected temperature is acquired from a temperature sensor before processes (a1) to (a7) are performed, and
wherein (b2) processes (a1) to (a7) are started on a basis of the acquired information about detected temperature.

### Claim 7 (depends on 1)

The camera parameter set calculation method according to claim 1, wherein (c1) information about detected impact is acquired from an impact sensor before processes (a1) to (a7) are performed, and
wherein (c2) processes (a1) to (a7) are started on a basis of the acquired information about detected impact.

### Claim 8 (depends on 1)

The camera parameter set calculation method according to claim 1, wherein (d1) information about time is acquired from a timer before processes (a1) to (a7) are performed, and
wherein (d2) processes (a1) to (a7) are started on a basis of the acquired information about time.

### Claim 9

A camera parameter set calculation method comprising:
(a1) acquiring a first image captured by a first camera and a second image captured by a second camera;
(a2) acquiring a first camera parameter set including one or more camera parameters of the first camera and a second camera parameter set including one or more camera parameters of the second camera;
(a3) repeating estimation of a pair of corresponding points that are a first corresponding point in the first image and a second corresponding point in the second image, to determine which points in the first image and in the second image indicates images of an identical point on the object, based on a pixel value at the first corresponding point and a pixel value at the second corresponding point, thereby pairs of corresponding points being provided;
(a4) calculating three-dimensional coordinate sets each corresponding to the pairs of corresponding points on a basis of the first camera parameter set and the second camera parameter set;
(a5) determining first pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the first image on a basis of the first camera parameter set and determining second pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the second image on a basis of the second camera parameter set;
(a6) calculating an evaluation value on a basis of pixel values at the first pixel coordinate pairs in the first image and pixel values at the second pixel coordinate pairs in the second image;
(a7) updating the first camera parameter set and the second camera parameter set on a basis of the evaluation value; and
(a8) outputting the updated first camera parameter set and second camera parameter set,
wherein at least one of processes (a1) to (a8) is performed by a processor,
the three-dimensional coordinate sets are represented by (x1, y1, z1) . . . (xi, yi, zi) . . . (xn, yn, zn), the first pixel coordinate pairs are represented by (u11, v11) . . . (u1i, v1i) . . . (u1n, vin), and the second pixel coordinate pairs are represented by (u21, v21) . . . (u2i, v2i) . . . (u2n, v2n), where 1<i≤n, and i and n are natural numbers,
the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u1i, v1i), the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u2i, v2i), the pixel value at the pixel coordinate pair (u1i, v1i) is represented by I1(u1i, v1i), and the pixel value at the pixel coordinate pair (u2i, v2i) is represented by I2(u2i, v2i), and
in process (a6), the evaluation value is calculated on a basis of a difference Di between the pixel values I1(u1i, v1i) and I2(u2i, v2i).

### Claim 10

A non-transitory computer-readable recording medium storing a control program executable by an apparatus including a processor, the control program comprising code to perform:
(a1) acquiring a first image captured by a first camera and a second image captured by a second camera;
(a2) acquiring a first camera parameter set including one or more camera parameters of the first camera and a second camera parameter set including one or more camera parameters of the second camera;
(a3) calculating three-dimensional coordinate sets on an object on a basis of the first image, the second image, the first camera parameter set, and the second camera parameter set, the first image including an image of the object, the second image including an image of the object;
(a4) determining first pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the first image on a basis of the first camera parameter set and determining second pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the second image on a basis of the second camera parameter set;
(a5) calculating an evaluation value on a basis of pixel values at the first pixel coordinate pairs in the first image and pixel values at the second pixel coordinate pairs in the second image;
(a6) updating the first camera parameter set and the second camera parameter set on a basis of the evaluation value; and
(a7) outputting the updated first camera parameter set and second camera parameter set,
wherein the three-dimensional coordinate sets are represented by (x1, y1, z1) . . . (xi, yi, zi) . . . (xn, yn, zn), the first pixel coordinate pairs are represented by (u11, v11) . . . (u1i, v1i) . . . (u1n, vin), and the second pixel coordinate pairs are represented by (u21, v21) . . . (u2i, v2i) . . . (u2n, v2n), where 1<i≤n, and i and n are natural numbers,
the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u1i, v1i), the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u2i, v2i), the pixel value at the pixel coordinate pair (u1i, v1i) is represented by I1(u1i, v1i), and the pixel value at the pixel coordinate pair (u2i, v2i) is represented by I2(u2i, v2i), and
in process (a5), the evaluation value is calculated on a basis of a difference Di between the pixel values I1(u1i, v1i) and I2(u2i, v2i).

### Claim 11

A camera parameter set calculation apparatus comprising a processing circuit that performs the following operations:
(a1) acquiring a first image captured by a first camera and a second image captured by a second camera;
(a2) acquiring a first camera parameter set including one or more camera parameters of the first camera and a second camera parameter set including one or more camera parameters of the second camera;
(a3) calculating three-dimensional coordinate sets on an object on a basis of the first image; the second image; the first camera parameter set; and the second camera parameter set, the first image including an image of the object, the second image including an image of the object;
(a4) determining first pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the first image on a basis of the first camera parameter set and determining second pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the second image on a basis of the second camera parameter set;
(a5) calculating an evaluation value on a basis of pixel values at the first pixel coordinate pairs in the first image and pixel values at the second pixel coordinate pairs in the second image;
(a6) updating the first camera parameter set and the second camera parameter set on a basis of the evaluation value; and
(a7) outputting the updated first camera parameter set and second camera parameter set,
wherein the three-dimensional coordinate sets are represented by (x1, y1, z1) . . . (xi, yi, zi) . . . (xn, yn, zn), the first pixel coordinate pairs are represented by (u11, v11) . . . (u1i, v1i) . . . (u1n, vin), and the second pixel coordinate pairs are represented by (u21, v21) . . . (u2i, v2i) . . . (u2n, v2n), where 1<i≤n, and i and n are natural numbers,
the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u1i, v1i), the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u2i, v2i), the pixel value at the pixel coordinate pair (u1i, v1i) is represented by I1(u1i, v1i), and the pixel value at the pixel coordinate pair (u2i, v2i) is represented by I2(u2i, v2i), and
in process (a5), the evaluation value is calculated on a basis of a difference Di between the pixel values I1(u1i, v1i) and I2(u2i, v2i).

### Claim 12

A camera parameter set calculation apparatus comprising a processing circuit that performs the following operations:
(a1) acquiring a first image captured by a first camera and a second image captured by a second camera;
(a2) acquiring a first camera parameter set including one or more camera parameters of the first camera and a second camera parameter set including one or more camera parameters of the second camera;
(a3) repeating estimation of a pair of corresponding points that are a first corresponding point in the first image and a second corresponding point in the second image, to determine which points in the first image and in the second image indicates images of an identical point on the object, based on a pixel value at the first corresponding point and a pixel value at the second corresponding point, thereby pairs of corresponding points being provided;
(a4) calculating three-dimensional coordinate sets each corresponding to the pairs of corresponding points on a basis of the first camera parameter set and the second camera parameter set;
(a5) determining first pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the first image on a basis of the first camera parameter set and determining second pixel coordinate pairs obtained by projecting the three-dimensional coordinate sets onto the second image on a basis of the second camera parameter set;
(a6) calculating an evaluation value on a basis of pixel values at the first pixel coordinate pairs in the first image and pixel values at the second pixel coordinate pairs in the second image;
(a7) updating the first camera parameter set and the second camera parameter set on a basis of the evaluation value; and
(a8) outputting the updated first camera parameter set and second camera parameter set,
wherein the three-dimensional coordinate sets are represented by (x1, y1, z1) . . . (xi, yi, zi) . . . (xn, yn, zn), the first pixel coordinate pairs are represented by (u11, v11) . . . (u1i, v1i) . . . (u1n, vin), and the second pixel coordinate pairs are represented by (u21, v21) . . . (u2i, v2i) . . . (u2n, v2n), where 1<i≤n, and i and n are natural numbers,
the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u1i, v1i), the three-dimensional coordinate set (xi, yi, zi) corresponds to the pixel coordinate pair (u2i, v2i), the pixel value at the pixel coordinate pair (u1i, v1i) is represented by I1(u1i, v1i), and the pixel value at the pixel coordinate pair (u2i, v2i) is represented by I2(u2i, v2i), and
in process (a6), the evaluation value is calculated on a basis of a difference Di between the pixel values I1(u1i, v1i) and I2(u2i, v2i).