Multi-sensor collaborative calibration system

A system for calibration of multiple sensors on or in a vehicle, the system comprising a processor configured to:

• obtain, from a first sensor and a second sensor located on a vehicle, sensor data items of an area that comprises a plurality of calibration objects;
• determine, from the sensor data items for each of the first sensor and the second sensor, attributes of the plurality of calibration objects;
• determine, for the first sensor and the second sensor, an initial matrix that describes a first set of extrinsic parameters between the first sensor and the second sensor based at least on the attributes of the plurality of calibration objects;
• determine an updated matrix that describes a second set of extrinsic parameters between the first sensor and the second sensor based at least on the initial matrix and a location of at least one calibration object, wherein the updated matrix is determined by using a factor graph that minimizes a summation of at least a first factor associated with the first sensor and a second factor associated with the second sensor, and
• wherein each the first factor and the second factor is based at least on:
• (1) the initial matrix,
• (2) coordinates of at least one calibration object obtained from the sensor data items for the first sensor and the second sensor,
• (3) a third set of extrinsic parameters that describe a location or a pose of the second sensor at a first time and another location or another pose of the second sensor at a second time that is later in time than the first time, and
• (4) the location of the at least one calibration object;
• and
• perform autonomous operation of the vehicle using the second set of extrinsic parameters and additional sensor data received from the first sensor and the second sensor when the vehicle is operated on a road.

A non-transitory computer readable storage medium having code stored thereon, the code, when executed by a processor, causing the processor to implement a method comprising:

• obtaining, from a first sensor and a second sensor located on a vehicle, sensor data items of an area that comprises a plurality of calibration objects;
• determining, from the sensor data items for each of the first sensor and the second sensor, attributes of the plurality of calibration objects;
• determining, for the first sensor and the second sensor, an initial matrix that describes a first set of extrinsic parameters between the first sensor and the second sensor based at least on the attributes of the plurality of calibration objects;
• determining an updated matrix that describes a second set of extrinsic parameters between the first sensor and the second sensor based at least on the initial matrix and a location of at least one calibration object, wherein the updated matrix is determined by using a factor graph that minimizes a summation of at least a first factor associated with the first sensor and a second factor associated with the second sensor, and
• wherein each the first factor and the second factor is based at least on:
• (1) the initial matrix,
• (2) coordinates of at least one calibration object obtained from the sensor data items for the first sensor and the second sensor,
• (3) a third set of extrinsic parameters that describe a location or a pose of the second sensor at a first time and another location or another pose of the second sensor at a second time that is later in time than the first time, and
• (4) the location of the at least one calibration object;
• and
• performing autonomous operation of the vehicle using the second set of extrinsic parameters and additional sensor data received from the first sensor and the second sensor when the vehicle is operated on a road.