# US 2021/0279845 A1 · Automatic fisheye camera calibration for video analytics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279845 A1
- Document ID: US20210279845A1
- Status: Published application
- Kind: A1
- Application number: 17/182,685
- Filed: 2021-02-23
- Published: 2021-09-09
- Earliest claimed: 2020-03-09
- Assignees: NEC Laboratories America, Inc.
- Applicants: NEC Laboratories America, Inc.
- Inventors: Min Feng; Srimat Chakradhar; Alper Yildirim
- Web page: https://patentgrep.com/patent/US20210279845A1

## CPC

- G06T 5/80: Geometric correction
- G06T 7/13: Edge detection
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 2207/10016: Video; Image sequence
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A computer-implemented method executed by at least one processor for reducing radial distortion errors in fish-eye images is presented. The method includes capturing an image from a camera including distortions, detecting arc-shaped edge segments in the image including the distortions, estimating a main distortion parameter by fixing a distortion centerpoint in a middle of the image, estimating the distortion centerpoint with the main distortion parameter, and obtaining an undistorted version of the captured image by inverting the distortion model.

## Claims

### Claim 1

A computer-implemented method executed by at least one processor for reducing radial distortion errors in fish-eye images, the method comprising:
capturing an image from a camera including distortions;
detecting arc-shaped edge segments in the image including the distortions;
estimating a main distortion parameter by fixing a distortion centerpoint in a middle of the image;
estimating the distortion centerpoint with the main distortion parameter; and
obtaining an undistorted version of the captured image by inverting the distortion model.

### Claim 2 (depends on 1)

The method of claim 1, wherein an area defined between a detected arc-shaped edge segment and a corresponding straight line segment or chord passing through both ends of the detected arc-shaped edge segment is minimized.

### Claim 3 (depends on 2)

The method of claim 2, wherein the area defined between the detected arc-shaped edge segment and the chord is a cost function.

### Claim 4 (depends on 3)

The method of claim 3, wherein the cost function is given as:
E
=
∑
k
=
1
N
⁢

∑
j
=
1
n
k
⁢
f
⁡
(
x
j
,
y
j
;
λ
,
x
o
,
y
o
)
d
k

where N is a number of circular arcs in the image, nk is a number of pixels for kth arc, dk is a distance between the endpoints of kth arc, and f(xj, yj, λ, xo, yo) is a signed distance between a point (xj, yj) and the chord under a set of distortion parameters of (λ, xo, yo).

### Claim 5 (depends on 3)

The method of claim 3, wherein, for each detected arc-shaped segment, a distance sum is divided by a length of the chord to obtain a normalized error between each detected arc-shaped segment and its corresponding chord, the distance sum defined as a sum of point-line distances for each detected arc-shaped segment.

### Claim 6 (depends on 1)

The method of claim 1, wherein the main distortion parameter is estimated by running an optimization procedure to obtain an optimum main distortion parameter value.

### Claim 7 (depends on 1)

The method of claim 1, wherein the undistorted version of the captured image is obtained from minimizing a geometric error.

### Claim 8

A system for applying rolling shutter (RS)-aware spatially varying differential homography fields for simultaneous RS distortion removal and image stitching, the system comprising:
a memory; and
a processor in communication with the memory, wherein the processor runs program code to:
capture an image from a camera including distortions;
detect arc-shaped edge segments in the image including the distortions;
estimate a main distortion parameter by fixing a distortion centerpoint in a middle of the image;
estimate the distortion centerpoint with the main distortion parameter; and
obtain an undistorted version of the captured image by inverting the distortion model.

### Claim 9 (depends on 8)

The system of claim 8, wherein an area defined between a detected arc-shaped edge segment and a corresponding straight line segment or chord passing through both ends of the detected arc-shaped edge segment is minimized.

### Claim 10 (depends on 9)

The system of claim 9, wherein the area defined between the detected arc-shaped edge segment and the chord is a cost function.

### Claim 11 (depends on 10)

The system of claim 10, wherein the cost function is given as:
E
=
∑
k
=
1
N
⁢

∑
j
=
1
n
k
⁢
f
⁡
(
x
j
,
y
j
;
λ
,
x
o
,
y
o
)
d
k

where N is a number of circular arcs in the image, nk is a number of pixels for kth arc, dk is a distance between the endpoints of kth arc, and f (xj, yj, λ, xo, yo) is a signed distance between a point (xj, yj) and the chord under a set of distortion parameters of (λ, xo, yo).

### Claim 12 (depends on 10)

The system of claim 10, wherein, for each detected arc-shaped segment, a distance sum is divided by a length of the chord to obtain a normalized error between each detected arc-shaped segment and its corresponding chord, the distance sum defined as a sum of point-line distances for each detected arc-shaped segment.

### Claim 13 (depends on 8)

The system of claim 8, wherein the main distortion parameter is estimated by running an optimization procedure to obtain an optimum main distortion parameter value.

### Claim 14 (depends on 8)

The system of claim 8, wherein the undistorted version of the captured image is obtained from minimizing a geometric error.

### Claim 15

A non-transitory computer-readable storage medium comprising a computer-readable program for applying rolling shutter (RS)-aware spatially varying differential homography fields for simultaneous RS distortion removal and image stitching, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:
capturing an image from a camera including distortions;
detecting arc-shaped edge segments in the image including the distortions;
estimating a main distortion parameter by fixing a distortion centerpoint in a middle of the image;
estimating the distortion centerpoint with the main distortion parameter; and
obtaining an undistorted version of the captured image by inverting the distortion model.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein an area defined between a detected arc-shaped edge segment and a corresponding straight line segment or chord passing through both ends of the detected arc-shaped edge segment is minimized.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein the area defined between the detected arc-shaped edge segment and the chord is a cost function.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the cost function is given as:
E
=
∑
k
=
1
N
⁢

∑
j
=
1
n
k
⁢
f
⁡
(
x
j
,
y
j
;
λ
,
x
o
,
y
o
)
d
k

where N is a number of circular arcs in the image, nk is a number of pixels for kth arc, dk is a distance between the endpoints of kth arc, and f (xj, yj, λ, xo, yo) is a signed distance between a point (xj, yj) and the chord under a set of distortion parameters of (λ, xo, yo).

### Claim 19 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein, for each detected arc-shaped segment, a distance sum is divided by a length of the chord to obtain a normalized error between each detected arc-shaped segment and its corresponding chord, the distance sum defined as a sum of point-line distances for each detected arc-shaped segment.

### Claim 20 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein the main distortion parameter is estimated by running an optimization procedure to obtain an optimum main distortion parameter value.