US 2020/0025931 A1 · Three-Dimensional Object Detection

Abstract

Generally, the disclosed systems and methods implement improved detection of objects in three-dimensional (3D) space. More particularly, an improved 3D object detection system can exploit continuous fusion of multiple sensors and/or integrated geographic prior map data to enhance effectiveness and robustness of object detection in applications such as autonomous driving. In some implementations, geographic prior data (e.g., geometric ground and/or semantic road features) can be exploited to enhance three-dimensional object detection for autonomous vehicle applications. In some implementations, object detection systems and methods can be improved based on dynamic utilization of multiple sensor modalities. More particularly, an improved 3D object detection system can exploit both LIDAR systems and cameras to perform very accurate localization of objects within three-dimensional space relative to an autonomous vehicle. For example, multi-sensor fusion can be implemented via continuous convolutions to fuse image data samples and LIDAR feature maps at different levels of resolution.

Claims20

Dependent claims

An object detection system, comprising:

a camera system configured to capture image data associated with an environment surrounding an autonomous vehicle;
a LIDAR system configured to capture LIDAR point cloud data associated with the environment surrounding the autonomous vehicle;
one or more processors;
one or more non-transitory computer-readable media that store:
a machine-learned neural network that comprises one or more fusion layers, wherein at least one of the one or more fusion layers is configured to fuse image features from the image data at a first level of resolution with LIDAR features from the LIDAR point cloud data at a second level of resolution that is different from the first level of resolution;
instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
generating, at the machine-learned neural network, a first data stream descriptive of the image data and a second data stream descriptive of the LIDAR point cloud data;
executing, at the machine-learned neural network, one or more continuous convolutions to fuse the image features from the first data stream with the LIDAR features from the second data stream;
and
generating a feature map comprising the fused image features from the first data stream and the LIDAR features from the second data stream.

7 more elements

Dependencies are inferred from the claim text.

Three-Dimensional Object Detection

Life of this patent

Three-Dimensional Object Detection

Life of this patent

Abstract

Claims20