Multi-sensor calibration system

A system for calibration of multiple sensors on or in a vehicle, the system comprising a processor configured to:

• obtain, from each of at least two sensors located on the vehicle, a sensor data item of a road comprising a lane marker;
• extract, from each sensor data item, a location information of the lane marker wherein the sensor data item is obtained as the vehicle is driven on the road, wherein the sensor data item from each of the at least two sensors are obtained and the location information of the lane marker are extracted from each sensor data item by the processor configured to:
• receive, from a camera located on the vehicle, an image comprising the lane marker;
• determine, from the image, locations of corners of the lane marker;
• receive, from a light detection and ranging (LiDAR) sensor located on the vehicle, a frame comprising point cloud data (PCD) of an area that comprises the lane marker;
• and
• determine a set of three-dimensional (3D) world coordinates of the corners of the lane marker, wherein the at least two sensors comprises the camera and the LiDAR sensor, wherein the set of 3D world coordinates of the corners of the lane marker are determined by the processor configured to:
• obtain, based on a location of the vehicle, a second set of previously stored three-dimension (3D) world coordinates of the corners of the lane marker;
• and
• obtain the set of 3D world coordinates of the corners of the lane marker in the frame by using the second set of previously stored 3D world coordinates of the corners of the lane marker and a coordinate system of the LiDAR sensor;
• and
• calculate extrinsic parameters of the at least two sensors based on determining a difference between the location information of the lane marker from each sensor data item and a previously stored location information of the lane marker, wherein the previously stored location information of the lane marker comprises the second set of previously stored 3D world coordinates of the corners of the lane marker.

A non-transitory computer readable storage medium having code stored thereon, the code, when executed by a processor, causing the processor to implement a method comprising:

• obtaining, from each of at least two sensors located on a vehicle, a sensor data item of a road comprising a lane marker;
• extracting, from each sensor data item, a location information of the lane marker, wherein the sensor data item is obtained as the vehicle is driven on the road, wherein the obtaining the sensor data item from each of the at least two sensors and the extracting the location information of the lane marker from each sensor data item comprises:
• receiving, from a first sensor located on the vehicle, a first sensor data comprising the lane marker;
• determining, from an image, locations of corners of the lane marker;
• receiving, from a second sensor located on the vehicle, a second sensor data of an area of the road that comprises the lane marker;
• and
• determining a set of three-dimensional (3D) world coordinates of the corners of the lane marker, wherein the set of 3D world coordinates of the corners of the lane marker are determined by:
• obtaining, based on a location of the vehicle, a second set of previously stored three-dimension (3D) world coordinates of the corners of the lane marker;
• and
• projecting the second set of previously stored 3D world coordinates of the corners of the lane marker to a coordinate system of the second sensor to obtain the set of 3D world coordinates of the corners of the lane marker in the second sensor data;
• and
• calculating extrinsic parameters of the at least two sensors based on determining a difference between the location information of the lane marker from each sensor data item and a previously stored location information of the lane marker, wherein the previously stored location information of the lane marker comprises the second set of previously stored 3D world coordinates of the corners of the lane marker.