# US 2018/0322657 A1 · Method and apparatus for automatic intrinsic camera calibration using images of a planar calibration pattern

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322657 A1
- Document ID: US20180322657A1
- Status: Published application
- Kind: A1
- Application number: 15/970,965
- Filed: 2018-05-04
- Published: 2018-11-08
- Earliest claimed: 2017-05-04
- Assignees: Second Spectrum, Inc.
- Applicants: Second Spectrum, Inc.
- Inventors: Charles Xavier Quentin Dubout; Horesh Beny Ben Shitrit
- Web page: https://patentgrep.com/patent/US20180322657A1

## CPC

- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 7/74: {involving reference images or patches}
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/757: {Matching configurations of points or features}
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30208: Marker matrix
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure relates to a high precision method, model, and apparatus for calibrating the intrinsic camera parameter values of one or more intrinsic parameters of a camera. The intrinsic parameters include focal lengths, skew, optical center, and radial distortion coefficients. The practical implementation of the solution consists of a camera capturing several images of a planar calibration pattern from different viewpoints, and a computer implemented processing pipeline. The solution does not require additional human input and is robust to misdetection of the calibration pattern in some of the images.

## Claims

### Claim 1

A method for automatic camera calibration of an image capture device, the method comprising:
obtaining, by a computing unit, a plurality of images of a planar calibration pattern captured by the image capture device from different respective angles;
detecting, by the computing unit, a plurality of distinguishable features depicted in the planar calibration pattern for each obtained image;
for each image, generating, by the computing unit, a respective lattice graph corresponding to the image based on the distinguishable features detected from the image, wherein each respective lattice graph has a plurality of nodes corresponding to distinguishable features;
repeatedly calibrating, by the computing unit, a plurality of intrinsic camera parameter values corresponding to one or more intrinsic camera parameters of the image capture device based on two or more distinguishable features detected in two or more of the images and the lattice graphs corresponding to the two or more images, wherein the calibrating is performed for two or more different combinations of the images;
selecting, by the computing unit, one or more intrinsic camera parameter values from the plurality of intrinsic camera parameter values, wherein the one or more intrinsic camera parameter values are selected to minimize a robust estimator of a reprojection error over the obtained plurality of images; and
refining, by the computing unit, calibration of the plurality of intrinsic camera parameter values based on the one or more selected intrinsic camera parameters.

### Claim 2 (depends on 1)

The method of claim 1, further comprising, repeated calibrating a plurality of extrinsic camera parameter values using two and three-dimensional correspondences for the calibration images and the plurality of intrinsic camera parameter values, wherein the repeated calibrating is performed for two or more different combinations of obtained images.

### Claim 3 (depends on 1)

The method of claim 1, wherein obtaining the plurality of images includes obtaining a video containing the plurality of images.

### Claim 4 (depends on 1)

The method of claim 1, wherein the plurality of images is a series of standstill images.

### Claim 5 (depends on 1)

The method of claim 1, wherein the plurality of images is captured by the image capture device, wherein the computing unit is contained in a form factor of the image capture device.

### Claim 6 (depends on 1)

The method of claim 1, wherein the plurality of images is captured by the image capture device and received by the computing unit via a network.

### Claim 7 (depends on 1)

The method of claim 1, wherein detecting the plurality of distinguishable features includes, for each image, performing a blob detection process on the image.

### Claim 8 (depends on 7)

The method of claim 7, wherein the blob detection process is Laplacian of Gaussian.

### Claim 9 (depends on 7)

The method of claim 7, wherein the blob detection process is Maximally Stable Extremum Regions.

### Claim 10 (depends on 1)

The method of claim 1, wherein each node of the lattice graph is based on a respective center of a respective distinguishable feature of the plurality of distinguishable features.

### Claim 11 (depends on 1)

The method of claim 1, wherein refining the plurality of intrinsic camera parameter values is further based on all images of the plurality of obtained images that correspond to the one or more selected intrinsic camera parameter values.

### Claim 12 (depends on 1)

The method of claim 1, further comprising configuring the one or more intrinsic camera parameters of the image capture device in accordance with the refined plurality of intrinsic camera parameter values.

### Claim 13 (depends on 1)

The method of claim 1, wherein the one or more intrinsic camera parameters include focal lengths, skew, optical center, and/or radial distortion coefficients.

### Claim 14

A computing unit configured for automatic camera calibration of an image capture device, the computing unit comprising:
memory storing a plurality of computer-executable instructions;
one or more processors that execute the computer-executable instructions, the computer-executable instructions causing the one or more processors to:
obtain a plurality of images of a planar calibration pattern captured by the image capture device from different respective angles;
detect a plurality of distinguishable features depicted in the planar calibration pattern for each obtained image;
generate a lattice graph having a plurality of nodes based on the distinguishable features detected from each image;
repeatedly calibrate a plurality of intrinsic camera parameter values corresponding to one or more intrinsic camera parameters of the image capture device based on two or more distinguishable features detected in two or more of the images and the lattice graphs corresponding to the two or more images, wherein the calibrating is performed for two or more different combinations of the images;
select, one or more intrinsic camera parameter values from the plurality of intrinsic camera parameter values, wherein the one or more intrinsic camera parameter values are selected to minimize a robust estimator of a reprojection error over the obtained plurality of images; and
refine calibration of the plurality of intrinsic camera parameters based on the one or more selected intrinsic camera parameters.

### Claim 15 (depends on 14)

The computing unit of claim 14, wherein obtaining the plurality of images includes obtaining a video containing the plurality of images.

### Claim 16 (depends on 14)

The computing unit of claim 14, wherein the plurality of images is a series of standstill images.

### Claim 17 (depends on 14)

The computing unit of claim 14, wherein the plurality of images is captured by the image capture device, and wherein the computing unit is integrated in a form factor of the image capture device.

### Claim 18 (depends on 14)

The computing unit of claim 14, wherein the plurality of images is captured by the image capture device and received by the computing unit via a network.

### Claim 19 (depends on 14)

The computing unit of claim 14, wherein detecting the plurality of distinguishable features includes, for each image, performing a blob detection process on the image.

### Claim 20 (depends on 19)

The computing unit of claim 19, wherein the blob detection process is Laplacian of Gaussian.

### Claim 21 (depends on 19)

The computing unit of claim 19, wherein the blob detection process is Maximally Stable Extremum Regions.

### Claim 22 (depends on 21)

The computing unit of claim 21, wherein each node of the lattice graph is based on a respective center of a respective distinguishable feature of the plurality of distinguishable features.

### Claim 23 (depends on 21)

The computing unit of claim 21, wherein refining the plurality of intrinsic camera parameter values is further based on all images of the plurality of obtained images that correspond to the one or more selected intrinsic camera parameter values.

### Claim 24 (depends on 14)

The computing unit of claim 14, wherein the computer-executable instructions further cause the one or more processors to configure the one or more intrinsic camera parameters of the image capture device in accordance with the refined plurality of intrinsic camera parameter values.

### Claim 25 (depends on 14)

The computing unit of claim 14, wherein the computer-executable instructions further cause the one or more processors to repeatedly calibrate a plurality of extrinsic camera parameter values of one or more extrinsic camera parameters of the image capture device using two and three-dimensional correspondences for the calibration images and the plurality of intrinsic camera parameter values, wherein the repeated calibrating is performed for two or more different combinations of obtained images.

### Claim 26 (depends on 14)

The computing unit of claim 14, wherein the one or more intrinsic camera parameters include focal lengths, skew, optical center, and/or radial distortion coefficients.
