US 9,578,310 B2Granted

Automatic scene calibration

AssigneeSoftkinetic Software

InventorsXavier Baele, Javier Martinez Gonzalez

Life of this patent

Estimating the term…

Loading patent…

Abstract

Described herein is a method of calibrating a three-dimensional imaging system. During calibration, a position and an orientation of the three-dimensional imaging system is determined with respect to a first parameter comprising a real world vertical direction (Vw) and to a second parameter comprising an origin of a three-dimensional scene captured by the imaging system. The first and second parameters are used to derive a calibration matrix (MC2w) which is used to convert measurements from a virtual coordinate system (Mc) of the three-dimensional imaging system into a real coordinate system (Mw) related to the real world. The calibration matrix (MC2w) is used to rectify measurements prior to signal processing. An inverse calibration matrix (Mw2c) is also determined. Continuous monitoring and adjustment of the setup of the three-dimensional imaging system is carried out and the calibration matrix (Mc2w) and its inverse (Mw2c) are adjusted accordingly.

Claims15

Dependent claims

A method of calibrating a three-dimensional time-of-flight imaging system in a three-dimensional environment, the method comprising the steps of:

a) determining a reference orthogonal virtual three-dimensional coordinate system for the three-dimensional imaging system, the reference orthogonal virtual three-dimensional coordinate system having horizontal, vertical and depth axes in which the horizontal axis and the vertical axis are respectively aligned with the horizontal and vertical axes of a sensor in the three-dimensional imaging system with the depth axis being orthonormal to the plane of the sensor defined by its horizontal and vertical axes;
b) obtaining a vertical direction of the real world in the virtual coordinate system;
c) determining with respect to the reference coordinate system a real world three-dimensional orthonormal coordinate system having horizontal, vertical and depth axes in which the vertical axis is rotated to align it with respect to the vertical direction;
d) determining a point in the scene as a new origin for the real world three-dimensional orthonormal coordinate system;
e) deriving a translation vector from the origin of the virtual three-dimensional coordinate system to the point defined as the new origin of the scene;
f) deriving a rotation matrix for transforming the virtual three-dimensional coordinate system into the real world three-dimensional coordinate system;
g) deriving a calibration matrix (Mc2w) for the three-dimensional imaging system as the rotation matrix translated by the translation vector;
h) aligning a plane defined by the vertical and depth axes of the real world three-dimensional coordinate system to be coplanar with a plane defined by the virtual vertical axis and the virtual depth axis of the virtual three-dimensional coordinate system;
and
i) automatically refining the calibration matrix if changes are detected in at least one of:
the position and the orientation of the three-dimensional imaging system with respect to the three-dimensional environment.

8 more elements

A depth sensing imaging system having motorised means for adjusting at least one of:

position and orientation of the depth sensing imaging system in accordance with operation of a method comprising the steps of:
a) determining a reference orthogonal virtual three-dimensional coordinate system for the three-dimensional imaging system, the reference orthogonal virtual three-dimensional coordinate system having horizontal, vertical and depth axes in which the horizontal axis and the vertical axis are respectively aligned with the horizontal and vertical axes of a sensor in the three-dimensional imaging system with the depth axis being orthonormal to the plane of the sensor defined by its horizontal and vertical axes;
b) obtaining a vertical direction of the real world in the virtual coordinate system:
c) determining with respect to the reference coordinate system a real world three-dimensional orthonormal coordinate system having horizontal, vertical and depth axes in which the vertical axis is rotated to align it with respect to the vertical direction;
d) determining a point in the scene as a new origin for the real world three-dimensional orthonormal coordinate system;
e) deriving a translation vector from the origin of the virtual three-dimensional coordinate system to the point defined as the new origin of the scene;
f) deriving a rotation matrix for transforming the virtual three-dimensional coordinate system into the real world three-dimensional coordinate system;
g) deriving a calibration matrix for the three-dimensional imaging system as the rotation matrix translated by the translation vector;
h) aligning a plane defined by the vertical and depth axes of the real world three-dimensional coordinate system to be coplanar with a plane defined by the virtual vertical axis and the virtual depth axis of the virtual three-dimensional coordinate system;
and
i) automatically refining the calibration matrix if changes are detected in at least one of:
the position and the orientation of the three-dimensional imaging system with respect to the three-dimensional environment.

9 more elements

A three-dimensional time of flight imaging system having calibration means operating in accordance with a method comprising the steps of:

a) determining a reference orthogonal virtual three-dimensional coordinate system for the three-dimensional imaging system, the reference orthogonal virtual three-dimensional coordinate system having horizontal, vertical and depth axes in which the horizontal axis and the vertical axis are respectively aligned with the horizontal and vertical axes of a sensor in the three-dimensional imaging system with the depth axis being orthonormal to the plane of the sensor defined by its horizontal and vertical axes;
b) obtaining a vertical direction of the real world in the virtual coordinate system:
c) determining with respect to the reference coordinate system a real world three-dimensional orthonormal coordinate system having horizontal, vertical and depth axes in which the vertical axis is rotated to align it with respect to the vertical direction;
d) determining a point in the scene as a new origin for the real world three-dimensional orthonormal coordinate system;
e) deriving a translation vector from the origin of the virtual three-dimensional coordinate system to the point defined as the new origin of the scene;
f) deriving a rotation matrix for transforming the virtual three-dimensional coordinate system into the real world three-dimensional coordinate system;
g) deriving a calibration matrix for the three-dimensional imaging system as the rotation matrix translated by the translation vector;
h) aligning a plane defined by the vertical and depth axes of the real world three-dimensional coordinate system to be coplanar with a plane defined by the virtual vertical axis and the virtual depth axis of the virtual three-dimensional coordinate system;
and
i) automatically refining the calibration matrix if changes are detected in at least one of:
the position and the orientation of the three-dimensional imaging system with respect to the three-dimensional environment.

8 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/130,256Filed 14 Jan 2013
Priority: EP 1200020213 Jan 2012
Published: US 2015/0181198 A125 Jun 2015
Granted: 21 Feb 2017
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

H04N 13/246
Using stereoscopic image cameras: calibration of cameras
G06T 7/50
Image analysis: depth or shape recovery
G06T 7/80
Image analysis: analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
G06T 7/85
Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration: stereo camera calibration