US 2019/0096091 A1Published application

Systems and methods to improve camera intrinsic parameter calibration

AssigneeBaidu USA LLC

InventorsYingze BAO, Mingyu CHEN

Life of this patent

Estimating the term…

Loading patent…

Abstract

Described herein are systems and methods that provide easy and effective camera calibration. In one or more embodiments, a set of calibration patterns, such as array of unique markers, are used as a calibration target. The unique calibration markers resolve ambiguity when only partial views of the calibration target are captured. Embodiments disclosed herein also allow for an interactive calibration process that can direct users to specific locations within the camera image that require additional calibration image captures. Also, in one or more embodiments, the calibration process may be checked at one or more stages to help insure that the final camera intrinsic parameters will be sufficiently accuracy. One of the verifications may include addressing overfitting by, for example, using a first subset of the captured images to compute the intrinsic parameters and using a second set of the captured images to verify those intrinsic parameters.

Claims20

Dependent claims

A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by one or more processors, causes steps for performing intrinsic parameter calibration of a camera comprising:

capturing a set of captured images of a calibration target, the calibration target comprising a set of unique calibration markers and is positioned so that the camera captures a view of at least a portion of the calibration target in an image frame of the camera;
dividing the image frame of the camera into regions;
for each captured image of the set of captured images, detecting one or more features of one or more calibration markers of the calibration target that are viewable in the captured image;
counting a number of detected features for each region of the image frame;
and
responsive to the regions containing a number of detected features above a detected features threshold:
selecting images from the set of captured images to form a set of selected images;
responsive to the set of selected images having detected features at or above a detected features threshold for each region of the image frame, selecting a subset of detected features to determine an estimate of intrinsic calibration parameters of the camera;
using the subset of detected features to estimate the intrinsic calibration parameters of the camera;
re-projecting feature points of images from a set of verification images to verify the estimated intrinsic calibration parameters of the camera;
computing a re-projecting error for the re-projected feature points for each image from the set of verification images;
responsive to the re-projection error of an image being less than an error threshold, denoting the image as an inlier, otherwise denoting the image as an outlier;
and
responsive to the number of inlier images being greater than a first inlier threshold value, using the inliers to compute the intrinsic calibration parameters as final intrinsic calibration parameters for the camera.

11 more elements

A processor-implemented method for performing intrinsic parameter calibration of a camera comprising:

capturing a set of captured images of a calibration target, the calibration target comprising a set of unique calibration markers and is positioned so that the camera captures a view of at least a portion of the calibration target in an image frame of the camera;
dividing the image frame of the camera into regions;
for each captured image of the set of captured images, detecting one or more features of one or more calibration markers of the calibration target that are viewable in the captured image;
counting a number of detected features for each region of the image frame;
and
responsive to the regions containing a number of detected features above a detected features threshold:
selecting images from the set of captured images to form a set of selected images;
responsive to the set of selected images having detected features at or above a detected features threshold for each region of the image frame, selecting a subset of detected features to determine an estimate of intrinsic calibration parameters of the camera;
using the subset of detected features to estimate the intrinsic calibration parameters of the camera;
re-projecting feature points of images from a set of verification images to verify the estimated intrinsic calibration parameters of the camera;
computing a re-projecting error for the re-projected feature points for each image from the set of verification images;
responsive to the re-projection error of an image being less than an error threshold, denoting the image as an inlier, otherwise denoting the image as an outlier;
and
responsive to the number of inlier images being greater than a first inlier threshold value, using the inliers to compute the intrinsic calibration parameters as final intrinsic calibration parameters for the camera.

11 more elements

A camera system comprising:

an imaging system for capturing an image of a calibration target, the calibration target comprising a set of unique calibration makers;
one or more processors;
and
a non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by at least one of the one or more processors, causes steps for performing intrinsic parameter calibration of a camera comprising:
capturing a set of captured images of a calibration target, the calibration target comprising a set of unique calibration markers and is positioned so that the camera captures a view of at least a portion of the calibration target in an image frame of the camera;
dividing the image frame of the camera into regions;
for each captured image of the set of captured images, detecting one or more features of one or more calibration markers of the calibration target that are viewable in the captured image;
counting a number of detected features for each region of the image frame;
and
responsive to the regions containing a number of detected features above a detected features threshold:
selecting images from the set of captured images to form a set of selected images;
responsive to the set of selected images having detected features at or above a detected features threshold for each region of the image frame,
selecting a subset of detected features to determine an estimate of intrinsic calibration parameters of the camera;
using the subset of detected features to estimate the intrinsic calibration parameters of the camera;
re-projecting feature points of images from a set of verification images to verify the estimated intrinsic calibration parameters of the camera;
computing a re-projecting error for the re-projected feature points for each image from the set of verification images;
responsive to the re-projection error of an image being less than an error threshold, denoting the image as an inlier, otherwise denoting the image as an outlier;
and
responsive to the number of inlier images being greater than a first inlier threshold value, using the inliers to compute the intrinsic calibration parameters as final intrinsic calibration parameters for the camera.

16 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/102,644Filed 13 Aug 2018
Priority: 28 Sep 2017
Published: 28 Mar 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification6 codes

G06T 7/85
Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration: stereo camera calibration
G06T 7/11
Segmentation; Edge detection: region-based segmentation
G06T 2207/20021
Special algorithmic details: dividing image into blocks, subimages or windows
G06T 2207/30204
Subject of image; Context of image processing: marker