# US 2020/0402260 A1 · Camera Calibration and/or Use of a Calibrated Camera

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402260 A1
- Document ID: US20200402260A1
- Status: Published application
- Kind: A1
- Application number: 16/975,939
- Filed: 2018-03-14
- Published: 2020-12-24
- Earliest claimed: 2018-03-14
- Assignees: Nokia Technologies Oy
- Applicants: Nokia Technologies Oy
- Inventors: Martin SCHRADER; Radu Ciprian BILCU; Adrian BURIAN
- Web page: https://patentgrep.com/patent/US20200402260A1

## CPC

- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 7/50: Depth or shape recovery
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An apparatus that calibrates a parametric mapping that maps between object points and image points. The apparatus captures an image of a calibration pattern including features defining object points. The apparatus determines, from the image, measured image points that correspond to the object points. The apparatus determines, from the mapping, putative image points that correspond to the object points. The apparatus minimizes a cumulative cost function dependent upon differences between the measured image points and putative image points to determine parameters of the parametric mapping. The mapping uses a parametric function to specify points where light rays travelling from object points to image points cross the optical axis.

## Claims

### Claim 1

An apparatus comprising circuitry configured to calibrate a parametric mapping that maps between object points and image points, comprising:
capturing an image of a calibration pattern comprising features defining object points; determining, from the image, measured image points that correspond to the object points;
determining, from the mapping, putative image points that correspond to the object points;
minimizing a cumulative cost function dependent upon differences between the measured image points and putative image points to determine parameters of the parametric mapping,
wherein the mapping uses a parametric function to specify points where light rays travelling from object points to image points cross the optical axis.

### Claim 2 (depends on 1)

An apparatus as claimed in claim 1, wherein the parametric function is a non-linear function of an angle the light ray travelling from an object point towards an image point makes with the optical axis as it crosses the optical axis.

### Claim 3 (depends on 1)

An apparatus as claimed in claim 1, wherein the parametric function comprises a polynomial function in the angle the light ray travelling from an object point towards an image point makes with the optical axis as it crosses the optical axis.

### Claim 4 (depends on 1)

An apparatus as claimed in claim 1, wherein an object point is positioned in three-dimensions by a displacement along the optical axis and a displacement from the optical axis and wherein the mapping transforms displacement along the optical axis by adding an offset defined by the parametric function.

### Claim 5 (depends on 1)

An apparatus as claimed in claim 1, wherein the mapping uses a parametric function to specify a viewpoint location, on the optical axis via an entrance pupil, of an object point.

### Claim 6 (depends on 5)

An apparatus as claimed in any claim 5, wherein the entrance pupil is variable in size.

### Claim 7 (depends on 5)

An apparatus as claimed in claim 5, wherein the mapping uses a parametric function to specify variation of an entrance pupil position along an optical axis with off-axis angle of incident light.

### Claim 8 (depends on 1)

An apparatus as claimed in claim 1, comprising a dioptric optical system.

### Claim 9 (depends on 1)

An apparatus as claimed in claim 1, wherein the field of view of the camera is greater than 60°.

### Claim 10

A method of calibrating a camera by calibrating a parametric mapping that maps between object points and image points, the method comprising:
capturing an image of a calibration pattern comprising features defining object points;
from the image, determining measured image points that correspond to the object points;
from the mapping, determining putative image points that correspond to the object points; and
minimizing a cumulative cost function dependent upon differences between the measured image points and putative image points to determine parameters of the parametric mapping,
wherein the mapping uses a parametric function to specify points where light rays travelling from object points to image points cross the optical axis.

### Claim 11 (depends on 10)

A method as claimed in claim 10, wherein the parametric function is a non-linear function of an angle the light ray travelling from an object point towards an image point makes with the optical axis as it crosses the optical axis.

### Claim 12 (depends on 10)

A method as claimed in claim 10, wherein the parametric function comprises a polynomial function in the angle the light ray travelling from an object point towards an image point makes with the optical axis as it crosses the optical axis.

### Claim 13 (depends on 10)

A method as claimed in claim 10, wherein an object point is positioned in three-dimensions by a displacement along the optical axis and a displacement from the optical axis and wherein the mapping transforms displacement along the optical axis by adding an offset defined by the parametric function.

### Claim 14 (depends on 10)

A method as claimed in claim 10, wherein the mapping uses a parametric function to specify a viewpoint location, on the optical axis via an entrance pupil, of an object point.

### Claim 15 (depends on 14)

A method as claimed in any claim 14, wherein the entrance pupil is variable in size.

### Claim 16 (depends on 14)

A method as claimed in claim 14, wherein the mapping uses a parametric function to specify variation of an entrance pupil position along an optical axis with off-axis angle of incident light.

### Claim 17 (depends on 10)

A method as claimed in claim 10, wherein the image of the calibration is captured via a dioptric optical system.

### Claim 18 (depends on 10)

A method as claimed in claim 10, wherein the field of view of the camera is greater than 120° or greater than 150°.

### Claim 19

(canceled)

### Claim 20

An apparatus comprising:
at least one processor; and
at least one non-transitory memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform calibrating a parametric mapping that maps between object points and image points by performing:
determining, from a captured image of a calibration pattern comprising features defining object points, measured image points that correspond to the object points; determining, from the mapping, putative image points that correspond to the object points;
minimizing a cumulative cost function dependent upon differences between the measured image points and putative image points to determine parameters of the parametric mapping,
wherein the mapping uses a parametric function to specify points where light rays travelling from object points to image points cross the optical axis.

### Claim at position 21

21-25. (canceled)

### Claim 26

A non-transitory program storage device comprising a computer program that when loaded by a processor enables:
determining, from an image captured using a calibrated camera, image points that correspond to object points; and
using a parametric mapping that maps between object points and image points using a parametric function to specify points where light rays travelling from object points to image points cross the optical axis to perform one or more of:
determine putative image points that correspond to the object points;
modify the image;
determine a measurement based on object points; or
determine a position of the camera.

### Claim 27

(canceled)

### Claim 28 (depends on 26)

The non-transitory program storage device as in claim 26 where the computer program, that when loaded by a processor, enables:
calibrating the parametric mapping that maps between the object points and the image points by capturing the image of a calibration pattern comprising features defining the object points;
determining, from the image of the calibration pattern comprising the features defining the object points that has been captured by the camera, measured image points that correspond to the object points;
minimizing a cumulative cost function dependent upon differences between the measured image points and putative image points to determine parameters of the parametric mapping,
wherein the mapping uses a parametric function to specify points where light rays travelling from object points to image points cross the optical axis.
