Systems and methods for two-stage 3d object detection network for sensor fusion

Abstract

Systems and methods are provided that implement two-stage three-dimensional (3D) object detection, which includes a distinct fusion of radar data and camera data. The radar data includes a four-dimensional (4D) millimeter-wave (MMW) radar point cloud, and the camera data includes a high-resolution image in the two-dimensional space (2D). Thereafter, a 3D ROI proposal is fused with 2D image data generating a 2D proposal projection. The 2D proposal projection comprises proposals that predict the position of objects in the high-resolution image. In some cases, proposals in the 2D proposal projection correspond to anchors in the 3D ROI proposal. Then, 2D object detection results are fused with the 3D ROI proposal to generate 3D object detection results. Two-stage 3D object detection allows vehicles to detect objects in a 3D space of the driving environment to navigate safely, avoid obstacles, and respond to dynamic changes in the surrounding environment while operating autonomously.

Dependencies are inferred from the claim text.