US 2020/0025935 A1 · Three-Dimensional Object Detection

Abstract

Generally, the disclosed systems and methods implement improved detection of objects in three-dimensional (3D) space. More particularly, an improved 3D object detection system can exploit continuous fusion of multiple sensors and/or integrated geographic prior map data to enhance effectiveness and robustness of object detection in applications such as autonomous driving. In some implementations, geographic prior data (e.g., geometric ground and/or semantic road features) can be exploited to enhance three-dimensional object detection for autonomous vehicle applications. In some implementations, object detection systems and methods can be improved based on dynamic utilization of multiple sensor modalities. More particularly, an improved 3D object detection system can exploit both LIDAR systems and cameras to perform very accurate localization of objects within three-dimensional space relative to an autonomous vehicle. For example, multi-sensor fusion can be implemented via continuous convolutions to fuse image data samples and LIDAR feature maps at different levels of resolution.

Claims20

Dependent claims

An object detection system, comprising:

a camera system configured to capture image data associated with an environment surrounding an autonomous vehicle;
a LIDAR system configured to capture LIDAR point cloud data associated with the environment surrounding the autonomous vehicle;
a map system configured to provide geographic prior data regarding the environment surrounding the autonomous vehicle;
a fusion system configured to:
modify, based on the geographic prior data, the LIDAR point cloud data representing a three-dimensional view of the environment into map-modified LIDAR data having a plurality of layers, each layer representing a two-dimensional view of the environment;
fuse image features from the image data with LIDAR features from the map-modified LIDAR data;
and
generate a feature map comprising the fused image features and the LIDAR features;
and
a detector system configured to detect, based on the feature map, three-dimensional objects of interest within the environment.

7 more elements

A computer-implemented method, comprising:

obtaining, by a computing system comprising one or more computing devices, image data associated with an environment surrounding an autonomous vehicle;
obtaining, by the computing system, LIDAR point cloud data associated with the environment surrounding the autonomous vehicle;
projecting, by the computing system, the LIDAR point cloud data to a bird's eye view representation of the LIDAR point cloud data;
obtaining, by the computing system, geographic prior data regarding the environment surrounding the autonomous vehicle;
modifying, by the computing system, based on the geographic prior data, the bird's eye view representation of the LIDAR point cloud data into map-modified LIDAR data having a plurality of layers, each layer representing a two-dimensional view of the environment;
fusing, by the computing system, image features from the image data with LIDAR features from the map-modified LIDAR data;
generating, by the computing system, a feature map comprising the fused image features and LIDAR features;
and
detecting, by the computing system, based on the feature map, three-dimensional objects of interest within the environment.

6 more elements

Dependencies are inferred from the claim text.

Three-Dimensional Object Detection

Life of this patent

Three-Dimensional Object Detection

Life of this patent

Abstract

Claims20