Back to results
US 2024/0393451 A1Published application
Object detection using radar and lidar fusion
AssigneeMotional AD LLC
InventorsXiaoli Meng, Lubing Zhou, Karan Rajendra Shetti
Life of this patent
Estimating the term…
Abstract
Provided are methods for object detection using radar and lidar fusion, which can include generating clusters combining clusters of point clouds for radar and lidar, respectively, from which fused features are determined using a deep learning model. Systems and computer program products are also provided.
Claims19
A method comprising:
- accessing, with at least one processor, a first point cloud representing light detection and ranging (LiDAR) sensor data regarding environment of a vehicle;
- accessing, with the at least one processor, a second point cloud representing radar detection and ranging (RADAR) sensor data regarding the environment of the vehicle;
- generating, with the at least one processor, a first set of clusters of points in the first point cloud and a first set of anchor boxes;
- generating, with the at least one processor, a second set of clusters of points in the second point cloud and a second set of anchor boxes;
- generating, with the at least one processor, a third set of clusters and a third anchor box based on an association of the first set of the clusters and the second set of the clusters in the first set of anchor boxes and the second set of anchor boxes, wherein generating the third set of clusters includes creating fused LiDAR/RADAR-generated features using features obtained from each of a LiDAR branch and a RADAR branch in a network;
- generating, with the at least one processor and using the fused LiDAR/RADAR-generated features, at least one of an object label, a bounding box, or a velocity for each cluster in the third set of clusters based on a machine learning model;
- and
- causing, with the at least one processor, the vehicle to traverse the environment based on the environment based on at least one of the object label, the bounding box, or the velocity for each cluster in the third set of clusters.
5 more elements
A system comprising:
- at least one processor;
- and
- memory storing instructions that when executed by the at least one processor, cause the at least one processor to perform operations comprising:
- accessing a first point cloud representing light detection and ranging (LiDAR) sensor data regarding environment of a vehicle;
- accessing a second point cloud representing radar detection and ranging (RADAR) sensor data regarding the environment of the vehicle;
- generating a first set of clusters of points in the first point cloud and a first set of anchor boxes;
- generating a second set of clusters of points in the second point cloud and a second set of anchor boxes;
- generating a third set of clusters and a third anchor box based on an association of the first set of the clusters and the second set of the clusters in the first set of anchor boxes and the second set of anchor boxes, wherein generating the third set of clusters includes creating fused LiDAR/RADAR-generated features using features obtained from each of a LiDAR branch and a RADAR branch in a network;
- generating, using the fused LiDAR/RADAR-generated features, at least one of an object label, a bounding box, or a velocity for each cluster in the third set of clusters based on a machine learning model;
- and
- causing the vehicle to traverse the environment based on the environment based on at least one of the object label, the bounding box, or the velocity for each cluster in the third set of clusters.
8 more elements
A non-transitory, computer-readable storage medium having stored thereon instructions that when executed by one or more processors, cause the one or more processors to perform operations comprising:
- accessing a first point cloud representing light detection and ranging (LiDAR) sensor data regarding environment of a vehicle;
- accessing a second point cloud representing radar detection and ranging (RADAR) sensor data regarding the environment of the vehicle;
- generating a first set of clusters of points in the first point cloud and a first set of anchor boxes;
- generating a second set of clusters of points in the second point cloud and a second set of anchor boxes;
- generating a third set of clusters and a third anchor box based on an association of the first set of the clusters and the second set of the clusters in the first set of anchor boxes and the second set of anchor boxes, wherein generating the third set of clusters includes creating fused LiDAR/RADAR-generated features using features obtained from each of a LiDAR branch and a RADAR branch in a network;
- generating, using the fused LiDAR/RADAR-generated features, at least one of an object label, a bounding box, or a velocity for each cluster in the third set of clusters based on a machine learning model;
- and
- causing the vehicle to traverse the environment based on the environment based on at least one of the object label, the bounding box, or the velocity for each cluster in the third set of clusters.
5 more elements
Dependencies are inferred from the claim text.