High definition map fusion for 3D object detection

A system, comprising:

• at least one processor, and
• at least one non-transitory storage media storing instructions that, when executed by the at least one processor, cause the at least one processor to:
• obtain raster maps, vector maps, and point cloud data;
• extract features from the raster maps, vector maps, and point cloud data to generate respective bird's eye view (BEV) representations of raster map features, vector map features, and point cloud features, wherein generating the BEV representation of the vector map features comprises:
• transforming lane segments in the vector map to egocentric coordinates;
• learning the vector map features using a neural network;
• and
• assigning the vector map features to a pixel of the BEV representation of the vector map features, wherein the BEV representation of the vector map features is of a same dimension as the BEV representation of the point cloud features and the BEV representation of the raster map features;
• fuse the BEV representation of the raster map features, the BEV representation of the vector map features, and the BEV representation of the point cloud features into a fused BEV image;
• and
• detect objects in the fused BEV image.

At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to:

• obtain raster maps, vector maps, and point cloud data;
• extract features from the raster maps, vector maps, and point cloud data to generate respective bird's eye view (BEV) representations of raster map features, vector map features, and point cloud features, wherein generating the BEV representation of the vector map features comprises:
• transforming lane segments in the vector map to egocentric coordinates;
• learning the vector map features using a neural network;
• and
• assigning the vector map features to a pixel of the BEV representation of the vector map features, wherein the BEV representation of the vector map features is of a same dimension as the BEV representation of the point cloud features and the BEV representation of the raster map features;
• fuse the BEV representation of the raster map features, the BEV representation of the vector map features, and the BEV representation of the point cloud features into a fused BEV image;
• and
• detect objects in the fused BEV image.