# US 12,111,180 B2 · Calibration of sensor position offsets based on rotation and translation vectors for matched trajectories > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,111,180 B2 - Document ID: US12111180B2 - Status: Granted - Kind: B2 - Application number: 17/856,690 - Filed: 2022-07-01 - Published: 2023-01-05 - Granted: 2024-10-08 - Earliest claimed: 2021-07-01 - Assignees: Summer Robotics, Inc. - Applicants: Summer Robotics, Inc. - Inventors: Brian Alexander Paden - Web page: https://patentgrep.com/patent/US12111180B2 ## CPC - G01C 25/00: Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass (testing, calibrating or compensating compasses G01C17/38) - G01C 3/02: Details - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06T 2207/10016: Video; Image sequence - G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence - G06T 2207/10024: Color image - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30241: Trajectory - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Embodiments are directed to calibrating multi-view triangulation systems that perceive surfaces and objects based on reflections of one or more scanned laser beams that are continuously sensed by two or more sensors. In addition to sampling and triangulating points from a spline formed by an unbroken line trajectory of a laser beam, the calibration system samples and triangulates a corresponding velocity vector. Iterative reduction is performed on velocity vectors instead of points or splines. The velocity vector includes directions and magnitudes along a trajectory of a scanning laser beam which are used to determine the actual velocities. Translation and rotation vectors are based on the velocity vectors for matching trajectories determined for two or more sensors having offset physical positions, which are used to calibrate sensor offset errors associated with the matching trajectories provided to a modeling engine. ## Claims ### Claim 1 A method for perceiving surfaces using one or more processors that are configured to execute instructions, wherein the executed instructions enable actions, comprising: employing two or more sensors to separately monitor a continuous stream of sensor events based on one or more scanned light beam reflections from one or more surfaces; employing one or more portions of the monitored sensor events to generate one or more trajectories for each sensor; determining a set of velocity vectors associated with a trajectory for one of the two or more sensors that matches another set of determined velocity vectors associated with another trajectory; generating a translation vector and a rotation vector associated with the determined velocity vectors associated with the matching trajectories, wherein physical positions of the two or more sensors are offset from each other; calibrating the two or more sensors by decreasing one or more offset errors based on the translation vector and the rotation vector; and providing the matching trajectories, the translation vector, and the rotation vector to a modeling engine, wherein the modeling engine employs the matching trajectories to calculate a position of a surface in three dimensional space. ### Claim 2 (depends on 1) The method of claim 1, wherein each sensor event further comprises information that includes one or more of a timestamp, a column location, a row location, and a velocity. ### Claim 3 (depends on 1) The method of claim 1, wherein each trajectory is a parametric representation of a one-dimensional curve segment in a three-dimensional space. ### Claim 4 (depends on 1) The method of claim 1, further comprising: iteratively updating the one or more trajectories based on the continuous stream of sensor events; and recalibrating the one or more updated matching trajectories, wherein the one or more recalibrated trajectories are provided to the modeling engine. ### Claim 5 (depends on 1) The method of claim 1, further comprising: determining one or more objects based on a portion of the one or more trajectories that are associated with a portion of the one or more surfaces. ### Claim 6 (depends on 1) The method of claim 1, further comprising: providing a representation of the rotation vector in radians for each of an x, y and z axis; and providing a representation of the translation vector in magnitudes for each of an x, y, and z axis. ### Claim 7 A system for perceiving surfaces: a network computer, comprising: a memory that stores at least instructions; and one or more processors configured to execute instructions, wherein the execution of the instructions enables performance of actions, including: employing two or more sensors to separately monitor a continuous stream of sensor events based on one or more scanned light beam reflections from one or more surfaces; employing one or more portions of the monitored sensor events to generate one or more trajectories for each sensor; determining a set of velocity vectors associated with a trajectory for one of the two or more sensors that matches another set of determined velocity vectors associated with another trajectory; generating a translation vector and a rotation vector associated with the determined velocity vectors associated with the matching trajectories, wherein physical positions of the two or more sensors are offset from each other; calibrating the two or more sensors by decreasing one or more offset errors based on the translation vector and the rotation vector; and providing the matching trajectories, the translation vector, and the rotation vector to a modeling engine, wherein the modeling engine employs the matching trajectories to calculate a position of a surface in a three dimensional (3D) space; and one or more client computers, comprising: a memory that stores at least instructions; and one or more processors configured to execute instructions, wherein the executed instructions enable the performance of actions, including: providing one or more portions of the sensor events. ### Claim 8 (depends on 7) The system of claim 7, wherein each sensor event further comprises information that includes one or more of a timestamp, a column location, a row location, and a velocity. ### Claim 9 (depends on 7) The system of claim 7, wherein each trajectory is a parametric representation of a one-dimensional curve segment in a three-dimensional space. ### Claim 10 (depends on 7) The system of claim 7, further comprising: iteratively updating the one or more trajectories based on the continuous stream of sensor events; and recalibrating the one or more updated matching trajectories, wherein the one or more recalibrated trajectories are provided to the modeling engine. ### Claim 11 (depends on 7) The system of claim 7, further comprising: determining one or more objects based on a portion of the one or more trajectories that are associated with a portion of the one or more surfaces. ### Claim 12 (depends on 7) The system of claim 7, further comprising: providing a representation of the rotation vector in radians for each of an x, y and z axis; and providing a representation of the translation vector in magnitudes for each of an x, y, and z axis. ### Claim 13 A processor readable non-transitory storage media that includes instructions for perceiving surfaces, wherein execution of the instructions by one or more processors on one or more network computers enables performance of actions, comprising: employing two or more sensors to separately monitor a continuous stream of sensor events based on one or more scanned light beam reflections from one or more surfaces; employing one or more portions of the monitored sensor events to generate one or more trajectories for each sensor; determining a set of velocity vectors associated with a trajectory for one of the two or more sensors that matches another set of determined velocity vectors associated with another trajectory; generating a translation vector and a rotation vector associated with the determined velocity vectors associated with the matching trajectories, wherein physical positions of the two or more sensors are offset from each other; calibrating the two or more sensors by decreasing one or more offset errors based on the translation vector and the rotation vector; and providing the matching trajectories, the translation vector, and the rotation vector to a modeling engine, wherein the modeling engine employs the matching trajectories, translation vector and the rotation vector to calculate a position of a surface in a three dimensional (3D) space. ### Claim 14 (depends on 13) The processor readable non-transitory storage media of claim 13, wherein each sensor event further comprises information that includes one or more of a timestamp, a column location, a row location, and a velocity. ### Claim 15 (depends on 13) The processor readable non-transitory storage media of claim 13, wherein each trajectory is a parametric representation of a one-dimensional curve segment in a three-dimensional space. ### Claim 16 (depends on 13) The processor readable non-transitory storage media of claim 13, further comprising: iteratively updating the one or more trajectories based on the continuous stream of sensor events; and recalibrating the one or more updated matching trajectories, wherein the one or more recalibrated trajectories are provided to the modeling engine. ### Claim 17 (depends on 13) The processor readable non-transitory storage media of claim 13, further comprising: determining one or more objects based on a portion of the one or more trajectories that are associated with a portion of the one or more surfaces. ### Claim 18 (depends on 13) The processor readable non-transitory storage media of claim 13, further comprising: providing a representation of the rotation vector in radians for each of an x, y and z axis; and providing a representation of the translation vector in magnitudes for each of an x, y, and z axis.