Method and apparatus for camera-lidar calibration

An apparatus for camera LiDAR calibration, the apparatus comprising:

• a camera;
• a LiDAR;
• a processor;
• an acquiring device configured to receive an image captured by the camera at a specific time point and a LiDAR point cloud captured by the LiDAR and projected onto a camera coordinate system at the specific time point;
• an extracting device configured to extract a ground edge image corresponding to a edge of a ground surface, from the image, and to extract a road mark point cloud representing a point cloud for a road mark on the ground surface, from the LiDAR point cloud;
• a generating device configured to generate a first variation indicating a predicted position variation by predicting a position variation of the road mark point cloud to the ground edge image, through a neural network;
• a calibrating device configured to calibrate the camera and the LiDAR, based on the first variation,
• wherein the first variation comprises an element of a rotation matrix and an element of a translation matrix,
• wherein the calibrating device is configured to calibrate the camera and the LiDAR based on coordinates of the road mark point cloud and the first variation,
• wherein the acquiring device is configured to receive a training ground edge image and acquire a training road mark point cloud matched with the training ground edge image, and
• wherein the generating device is configured to:
• generate a training transformed point cloud by transforming the training road mark point cloud by a second variation, and
• generate a third variation indicating a predicted position variation by predicting a variation of a position of the training transformed point cloud to the training ground edge image through the neural network;
• and
• a training device configured to train the neural network by performing a regression analysis, based on the second variation and the third variation.

An apparatus for camera LiDAR calibration, the apparatus comprising:

• a camera;
• a LiDAR;
• a processing system including a processor and a non-transitory computer-readable storage medium storing a program to be executed by the processor, the program including instructions for:
• receiving an image captured by the camera at a specific time point;
• receiving a LiDAR point cloud captured by the LiDAR and projected onto a camera coordinate system at the specific time point;
• extracting from the image a ground edge image corresponding to an edge of a ground surface;
• extracting from the LiDAR point cloud a road mark point cloud representing a road mark on the ground surface;
• generating a first variation indicating a predicted position variation by predicting a position variation of the road mark point cloud to the ground edge image, through a neural network;
• calibrating the camera and the LiDAR, based on the first variation,
• wherein the neural network is trained based on:
• receiving a training ground edge image;
• receiving a training road mark point cloud matched with the training ground edge image;
• generating a training transformed point cloud by transforming the training road mark point cloud by a second variation;
• generating a third variation indicating a predicted position variation by predicting a variation of a position of the training transformed point cloud to the training ground edge image through the neural network;
• and
• training the neural network by performing a regression analysis, based on the second variation and the third variation,
• wherein the first variation comprises an element of a rotation matrix and an element of a translation matrix, and
• wherein the calibrating comprises calibrating the camera and the LiDAR based on coordinates of the road mark point cloud and the first variation.