US 10,999,511 B2 · LIDAR and camera synchronization

Abstract

A method and system for synchronizing a lidar and a camera on an autonomous vehicle. The system instructs the camera to detect light columns transmitted by the lidar. The system iterates through various start times for the camera. The system instructs the lidar to emit a plurality of light columns at a lidar frequency. The system instructs the camera to capture images at a camera frequency starting at each start time. The system analyzes the image data received from the cameras to identify light columns captured in the images. The system calculates an alignment score for each of the many start times based on the identified light columns. The start time with the optimal alignment score is selected and used to synchronize the lidar and the camera. With lidar data detected by the synchronized lidar and image data captured by the synchronized camera, the system may navigate the autonomous vehicle.

Claims30

Dependent claims

A method for synchronizing a light detection and ranging sensor (lidar) and a camera on an autonomous vehicle, comprising:

transmitting, via the lidar, a plurality of light columns at a lidar frequency within a field of view (FOV) of the camera;
for each start time of a plurality of start times:
capturing, via the camera, a plurality of images at a camera frequency, each image captured over an exposure time,
for each image of the plurality of images, identifying a number of light columns in the image, and
calculating an alignment score based on the identified number of light columns in each of the plurality of images;
selecting a start time with the highest alignment score;
synchronizing the lidar and the camera according to the selected start time;
and
navigating the autonomous vehicle with lidar data detected by the synchronized lidar and image data captured by the synchronized camera.

6 more elements

A non-transitory computer-readable storage medium for synchronizing a light detection and ranging sensor (lidar) and a camera on an autonomous vehicle, the non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:

transmitting, via the lidar, a plurality of light columns at a lidar frequency within a field of view (FOV) of the camera;
for each start time of a plurality of start times:
capturing, via the camera, a plurality of images at a camera frequency, each image captured over an exposure time,
for each image of the plurality of images, identifying a number of light columns in the image, and
calculating an alignment score based on the identified number of light columns in each of the plurality of images;
selecting a start time with the highest alignment score;
synchronizing the lidar and the camera according to the selected start time;
and
navigating the autonomous vehicle with lidar data detected by the synchronized lidar and image data captured by the synchronized camera.

6 more elements

A system comprising:

a lidar on an autonomous vehicle;
a camera on the autonomous vehicle;
a processor;
and
a non-transitory computer-readable storage medium for synchronizing a light detection and ranging sensor (lidar) and a camera on an autonomous vehicle, the non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
transmitting, via the lidar, a plurality of light columns at a lidar frequency within a field of view (FOV) of the camera;
for each start time of a plurality of start times:
capturing, via the camera, a plurality of images at a camera frequency, each image captured over an exposure time,
for each image of the plurality of images, identifying a number of light columns in the image, and
calculating an alignment score based on the identified number of light columns in each of the plurality of images;
selecting a start time with the highest alignment score;
synchronizing the lidar and the camera according to the selected start time;
and
navigating the autonomous vehicle with lidar data detected by the synchronized lidar and image data captured by the synchronized camera.

11 more elements

Dependencies are inferred from the claim text.

LIDAR and camera synchronization

Life of this patent

LIDAR and camera synchronization

Life of this patent

Abstract

Claims30