# US 10,371,512 B2 · Method and system for multiple 3D sensor calibration > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,371,512 B2 - Document ID: US10371512B2 - Status: Granted - Kind: B2 - Application number: 15/094,007 - Filed: 2016-04-08 - Published: 2017-10-12 - Granted: 2019-08-06 - Earliest claimed: 2016-04-08 - Assignees: Otis Elevator Company - Applicants: Otis Elevator Company - Inventors: Zhen Jia; Yanzhi Chen; Hui Fang; Arthur Hsu; Alan Matthew Finn - Web page: https://patentgrep.com/patent/US10371512B2 ## CPC - G01B 21/042: {Calibration or calibration artifacts (G01B3/30, G01B9/02072 take precedence)} - G01B 21/18: for measuring depth - G06T 7/85: {Stereo camera calibration} - B66B 5/0006: {Monitoring devices or performance analysers (B66B5/02 takes precedence)} - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30196: Human being; Person - G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - B66B: ELEVATORS; ESCALATORS OR MOVING WALKWAYS ({apparatus for raising or lowering persons on theatrical stages or the like A63J5/12} ; funicular railbound systems with rigid ground-supported tracks and cable traction, e.g. cliff railways, B61B9/00; arrangements of ammunition handlers in vessels B63G3/00; hoists, lifts, or conveyors for loading or unloading in general B65G; braking or detent devices controlling normal movements of winding drums or barrels B66D; ship-lifting devices E02C; garages for many vehicles with mechanical means for lifting vehicles E04H6/12; hoists for feeding ammunition or projectiles to launching apparatus or to loading mechanisms F41A9/00) ## Abstract A method for multiple 3D sensor calibration for a passenger conveyance system, the process including a computing a centroid location from spatial measurements for each of a pair of 3D sensors that form a common field of view via a moving object mathematical model; computing translation parameters from the locations of each centroid location; and correcting one of the pair of 3D sensors to a common world coordinate system from the translation parameters. ## Claims ### Claim 1 A method for multiple 3D sensor calibration without taking measurements in sensor and world coordinate systems, the process comprising: calculating a centroid location from spatial measurements for each of a pair of 3D sensors that form a common field of view via a moving object mathematical model; calculating at least one of translation and rotation parameters from the locations of each centroid location; synchronizing each of the 3D sensors in time; and correcting one of the pair of 3D sensors to a common world coordinate system from at least one of the translation and rotation parameters. ### Claim 2 (depends on 1) The method as recited in claim 1, wherein individually calibrating each of a pair of multiple 3D sensors to world coordinates results in zero relative rotation there between. ### Claim 3 (depends on 1) The method as recited in claim 1, further comprising synchronizing each of the multiple 3D sensors in time comprises synchronizing to within 1 frame time. ### Claim 4 (depends on 1) The method as recited in claim 1, further comprising calculating from the (x, y) locations of the two centroids (x1, y1) and (x2, y2), at least one translation parameter by subtraction. ### Claim 5 (depends on 1) The method as recited in claim 1, further comprising calculating from the (x, y) locations of the two centroids (x1, y1) and (x2, y2), at least one rotation parameter. ### Claim 6 (depends on 1) The method as recited in claim 1, further comprising calculating from the (x, y) locations of at least two centroids (x1,y1) and (x2,y2), the translation parameters (dx, dy) by a Least Squares (LS) solution. ### Claim 7 (depends on 1) The method as recited in claim 1, wherein one of the pair of 3D sensors is chosen as governing with respect to the common world coordinate system. ### Claim 8 (depends on 1) The method as recited in claim 1, further comprising individually calibrating each of a multiple 3D sensors to world coordinates. ### Claim 9 (depends on 1) The method as recited in claim 1, wherein three measurements are performed of a common point in response to three translation parameters being unknown. ### Claim 10 (depends on 1) The method as recited in claim 1, further comprising calculating the translation (dz) by finding individual ground planes of each sensor and computing a difference. ### Claim 11 (depends on 10) The method as recited in claim 10, further comprising using a random sample consensus (RANSAC) algorithm to find a consistent set of points to find each of the individual ground planes. ### Claim 12 (depends on 1) The method as recited in claim 1, wherein six measurements are performed of a common point in response to three rotation and three translation parameters being unknown. ### Claim 13 (depends on 12) The method as recited in claim 12, further comprising using a top and a bottom of a moving object and a separation between the top and the bottom of the moving object when the to three rotation and three translation parameters are unknown. ### Claim 14 (depends on 1) The method as recited in claim 1, wherein an object mathematical model is employed to estimate the common centroid location. ### Claim 15 (depends on 14) The method as recited in claim 14, wherein the object mathematical model is an elliptical cross-section mathematical model. ### Claim 16 (depends on 14) The method as recited in claim 14, wherein the object mathematical model is an elliptical cross-section person mathematical model. ### Claim 17 (depends on 14) The method as recited in claim 14, wherein the object mathematical model is a full 3D person shape mathematical model. ### Claim 18 (depends on 1) The method as recited in claim 1, wherein synchronizing each of the multiple 3D sensors in time comprises synchronizing to within 1 depth map sample time. ### Claim 19 (depends on 1) The method as recited in claim 1, wherein synchronizing each of the multiple 3D sensors in time comprises synchronizing to within 1/25th of a second. ### Claim 20 (depends on 1) The method as recited in claim 1, wherein synchronizing each of the multiple 3D sensors in time comprises synchronizing to within a depth map sample time small enough that moving objects are approximately stationary during the sample time. ### Claim 21 A sensor calibration system for a passenger conveyance system without taking measurements in both sensor and world coordinate systems, comprising: a multiple of sensors with at least one overlapping field of view; a processing module in communication with the multiple sensors, the processing module including a moving object mathematical model operable to estimate a common centroid location to compute translation parameters from the locations of each centroid location to facilitate correction of at least one of the pair of 3D sensors to a common world coordinate system from the translation parameters, each of the multiple of 3D sensors synchronized in time. ### Claim 22 (depends on 21) The system as recited in claim 21, wherein the object mathematical model is an elliptical cross-section mathematical model. ### Claim 23 (depends on 21) The system as recited in claim 21, wherein the object mathematical model is a full 3D person shape mathematical model. ### Claim 24 (depends on 21) The system as recited in claim 21, wherein each of the multiple of sensors generate a depth map in 1D, 2D, 3D, or a combination thereof.