Non-Repetitive Scanning Solid State LiDAR and Camera Extrinsic Calibration

A system for performing extrinsic calibration between a camera and a light detection and ranging (LiDAR) sensor, the system comprising:

• a processor;
• and
• a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
• receiving an image frame captured by the camera mounted on a device;
• receiving a point cloud captured by the LiDAR sensor mounted on the device, the LiDAR sensor having an overlapping field of view with the camera;
• identifying markers of a calibration target captured in the image frame by applying a feature identification model to the image frame;
• identifying the markers of the calibration target captured in the point cloud by:
• clustering points in the point cloud into one or more planes;
• selecting one of the planes based on sizes of the planes;
• and
• identifying holes in the selected plane as the markers of the calibration target;
• and
• determining a pose transformation between the camera and the LiDAR sensor based on information identifying the markers from the image frame and information identifying the markers from the point cloud.