Systems and Methods for Integrating Radar Data for Improved Object Detection in Autonomous Vehicles

A computing system for generating perception and prediction data for autonomous vehicles, the system comprising:

• one or more processors and one or more non-transitory computer-readable memories;
• wherein the one or more non-transitory computer-readable memories store instructions that, when executed by the processor, cause the computing system to perform operations, the operations comprising:
• accessing radar sensor data for an area around an autonomous vehicle, the radar sensor data including a plurality of radar points;
• accessing LIDAR sensor data for the area around the autonomous vehicle;
• generating, using one or more machine-learned models, a fused representation of the area around the autonomous vehicle based on the radar sensor data and the LIDAR data;
• determining, using the one or more machine-learned models, one or more objects in the area of the autonomous vehicle based on the fused representation;
• for a respective object in the one or more objects:
• selecting a plurality of radar points from the radar sensor data;
• generating, using a machine-learned model, a similarity score for each selected radar point;
• generating a weight associated with each radar point based on the similarity score;
• and
• calculating a predicted velocity for the respective object based on a weighted average of the velocities associated with the plurality of radar points;
• and
• generating a proposed motion plan based on the predicted velocity for the respective object.

An autonomous vehicle, comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
• accessing radar sensor data for an area around an autonomous vehicle, the radar sensor data including a plurality of radar points;
• accessing LIDAR sensor data for the area around the autonomous vehicle;
• generating, using one or more machine-learned models, a fused representation of the area around the autonomous vehicle based on the radar sensor data and the LIDAR data;
• determining, using the one or more machine-learned models, one or more objects in the area of the autonomous vehicle based on the fused representation;
• for a respective object in the one or more objects:
• selecting a plurality of radar points from the radar sensor data;
• generating, using a machine-learned model, a similarity score for each selected radar point;
• generating a weight associated with each radar point based on the similarity score;
• and
• calculating a predicted velocity for the respective object based on a weighted average of the velocities associated with the plurality of radar points;
• and
• generating a proposed motion plan based on the predicted velocity for the respective object.