# US 12,762,852 B2 · High definition map fusion for 3D object detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,852 B2 - Document ID: US12762852B2 - Status: Granted - Kind: B2 - Application number: 18/286,697 - Filed: 2023-04-14 - Published: 2026-01-22 - Granted: 2026-10-13 - Earliest claimed: 2023-04-14 - Assignees: Motional AD LLC - Applicants: Motional AD LLC - Inventors: Yiluan Guo; Abhinav Rai - Web page: https://patentgrep.com/patent/US12762852B2 ## CPC - G06V 10/806: {of extracted features} - G06V 10/764: using classification, e.g. of video objects - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 10/82: using neural networks - G06V 20/182: {Network patterns, e.g. roads or rivers} - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Provided are methods for high definition map fusions for 3D object detection. Some methods described also include obtaining, with at least one processor, raster maps, vector maps, and point cloud data and extracting, with the at least one processor, features from the raster maps, vector maps, and point cloud data to generate respective bird's eye view (BEV) representations. The methods also include fusing, with the at least one processor, the BEV representation of the raster map features, the BEV representation of the vector map features, and the BEV representation of the point cloud features into a fused BEV image. Additionally, the methods include detecting, with the at least one processor, objects in the fused BEV image. Systems and computer program products are also provided. ## Claims ### Claim 1 A method, comprising: obtaining, with at least one processor, raster maps, vector maps, and point cloud data; extracting, with the at least one processor, features from the raster maps, vector maps, and point cloud data to generate respective bird's eye view (BEV) representations of raster map features, vector map features, and point cloud features, wherein generating the BEV representation of the vector map features comprises: transforming lane segments in the vector map to egocentric coordinates; learning the vector map features using a neural network; and assigning the vector map features to a pixel of the BEV representation of the vector map features, wherein the BEV representation of the vector map features is of a same dimension as the BEV representation of the point cloud features and the BEV representation of the raster map features; fusing, with the at least one processor, the BEV representation of the raster map features, the BEV representation of the vector map features, and the BEV representation of the point cloud features into a fused BEV image; and detecting, with the at least one processor, objects in the fused BEV image. ### Claim 2 (depends on 1) The method of claim 1, wherein assigning the vector map features to a pixel of the BEV representation of the vector map features comprises assigning a value of zero for locations in the BEV representation of the vector map features without corresponding vector map features. ### Claim 3 (depends on 1) The method of claim 1, wherein the raster maps and vector maps are obtained from high definition maps. ### Claim 4 (depends on 1) The method of claim 1, wherein the raster map comprises a plurality of binary raster images. ### Claim 5 (depends on 1) The method of claim 1, wherein detecting objects in the fused BEV image comprises generating bounding boxes and track labels for each detected object. ### Claim 6 (depends on 1) The method of claim 1, wherein fusing the BEV representation of the raster map features, the BEV representation of the vector map features, and the BEV representation of the point cloud features into a fused BEV image is performed using concatenation, concatenation and masking, bitwise addition, or any combinations thereof. ### Claim 7 (depends on 1) The method of claim 1, comprising: obtaining the point cloud data from drive logs; labeling detected objects in corresponding drive logs; and training planning models using the labeled drive logs. ### Claim 8 A system, comprising: at least one processor, and at least one non-transitory storage media storing instructions that, when executed by the at least one processor, cause the at least one processor to: obtain raster maps, vector maps, and point cloud data; extract features from the raster maps, vector maps, and point cloud data to generate respective bird's eye view (BEV) representations of raster map features, vector map features, and point cloud features, wherein generating the BEV representation of the vector map features comprises: transforming lane segments in the vector map to egocentric coordinates; learning the vector map features using a neural network; and assigning the vector map features to a pixel of the BEV representation of the vector map features, wherein the BEV representation of the vector map features is of a same dimension as the BEV representation of the point cloud features and the BEV representation of the raster map features; fuse the BEV representation of the raster map features, the BEV representation of the vector map features, and the BEV representation of the point cloud features into a fused BEV image; and detect objects in the fused BEV image. ### Claim 9 (depends on 8) The system of claim 8, wherein assigning the vector map features to a pixel of the BEV representation of the vector map features comprises assigning a value of zero for locations in the BEV representation of the vector map features without corresponding vector map features. ### Claim 10 (depends on 8) The system of claim 8, wherein the raster maps and vector maps are obtained from high definition maps. ### Claim 11 (depends on 8) The system of claim 8, wherein the raster map comprises a plurality of binary raster images. ### Claim 12 (depends on 8) The system of claim 8, wherein detecting objects in the fused BEV image comprises generating bounding boxes and track labels for each detected object. ### Claim 13 (depends on 8) The system of claim 8, wherein fusing the BEV representation of the raster map features, the BEV representation of the vector map features, and the BEV representation of the point cloud features into a fused BEV image is performed using concatenation, concatenation and masking, bitwise addition, or any combinations thereof. ### Claim 14 (depends on 8) The system of claim 8, comprising: obtaining the point cloud data from drive logs; labeling detected objects in corresponding drive logs; and training planning models using the labeled drive logs. ### Claim 15 At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to: obtain raster maps, vector maps, and point cloud data; extract features from the raster maps, vector maps, and point cloud data to generate respective bird's eye view (BEV) representations of raster map features, vector map features, and point cloud features, wherein generating the BEV representation of the vector map features comprises: transforming lane segments in the vector map to egocentric coordinates; learning the vector map features using a neural network; and assigning the vector map features to a pixel of the BEV representation of the vector map features, wherein the BEV representation of the vector map features is of a same dimension as the BEV representation of the point cloud features and the BEV representation of the raster map features; fuse the BEV representation of the raster map features, the BEV representation of the vector map features, and the BEV representation of the point cloud features into a fused BEV image; and detect objects in the fused BEV image. ### Claim 16 (depends on 15) The at least one non-transitory storage media of claim 15, wherein assigning the vector map features to a pixel of the BEV representation of the vector map features comprises assigning a value of zero for locations in the BEV representation of the vector map features without corresponding vector map features. ### Claim 17 (depends on 15) The at least one non-transitory storage media of claim 15, wherein the raster maps and vector maps are obtained from high definition maps. ### Claim 18 (depends on 15) The at least one non-transitory storage media of claim 15, wherein the raster map comprises a plurality of binary raster images. ### Claim 19 (depends on 15) The at least one non-transitory storage media of claim 15, wherein detecting objects in the fused BEV image comprises generating bounding boxes and track labels for each detected object. ### Claim 20 (depends on 15) The at least one non-transitory storage media of claim 15, wherein fusing the BEV representation of the raster map features, the BEV representation of the vector map features, and the BEV representation of the point cloud features into a fused BEV image is performed using concatenation, concatenation and masking, bitwise addition, or any combinations thereof.