Sensor calibration using dense depth maps
AssigneeZoox, Inc.
InventorTill Kroeger
Life of this patent
Estimating the term…
Abstract
This disclosure is directed to calibrating sensors mounted on an autonomous vehicle. A dense depth map can be generated in a two-dimensional camera space using point cloud data generated by one of the sensors. Image data from another of the sensors can be compared to the dense depth map in the two-dimensional camera space. Differences determined by the comparison can indicate alignment errors between the sensors. Calibration data associated with the errors can be determined and used to calibrate the sensors without the need for calibration infrastructure.
Claims20
A system comprising:
- a vehicle configured to traverse through an environment;
- a sensor array disposed on the vehicle, the sensor array comprising a first LiDAR sensor having a first field of view and a second LiDAR sensor having a second field of view at least partially overlapping the first field of view;
- one or more processors;
- and
- computer-readable media storing instructions executable by the one or more processors, wherein the instructions cause the system to perform operations comprising:
- receiving, from the first LiDAR sensor, a first point cloud;
- receiving, from the second LiDAR sensor, a second point cloud;
- generating, based at least in part on the first point cloud, a first dense depth map in a two-dimensional space;
- generating, based at least in part on the second point cloud, a second dense depth map in the two-dimensional space;
- and
- determining, at a first pixel of a plurality of pixels in the two-dimensional space, a difference between a first value associated with a first parameter of the first dense depth map and a second value associated with a second parameter corresponding to the first parameter in the second dense depth map.
A method comprising:
- receiving, from a first sensor in an array of sensors and having a first field of view, point cloud data representing an environment;
- generating, based at least in part on the point cloud data, a dense depth map representing the environment, the dense depth map comprising a projection in a two-dimensional camera space;
- receiving, from a second sensor in the array of sensors and having a second field of view at least partially overlapping the first field of view, second data representing the environment;
- determining, in the two-dimensional camera space, a difference between a first parameter associated with the dense depth map and a second parameter associated with the second data;
- determining, based at least in part on the difference, a calibration function for the array of sensors, the calibration function representing at least one of a relative position or a relative orientation of the first sensor to the second sensor;
- and
- calibrating, based at least in part on the calibration function, first sensor data from the first sensor or second sensor data from the second sensor.
A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform operations comprising:
- receiving, from a first sensor in an array of sensors and having a first field of view, first sensor data representing an environment;
- receiving, from a second sensor in the array of sensors and having a second field of view at least partially overlapping the first field of view, second sensor data representing the environment;
- generated, based at least in part on the first sensor data and the second sensor data, a dense depth map comprising a projection in a two-dimensional camera space;
- determining, in the two-dimensional camera space, a difference between a first parameter associated with the dense depth map and a second parameter associated with the second sensor data;
- determining, based at least in part on the difference, a calibration function for the array of sensors, the calibration function representing a relative orientation of the first sensor and the second sensor;
- and
- calibrating, based at least in part on the calibration function, additional sensor data from the first sensor or the second sensor.
Dependencies are inferred from the claim text.