Radar and camera fusion for vehicle applications

An apparatus, comprising:

• a memory storing processor-readable code;
• and
• at least one processor coupled to the memory, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations including:
• receiving image Bird's Eye View (BEV) features;
• receiving first radio detection and ranging (RADAR) BEV features that are determined based on first RADAR data associated with a first data type;
• determining normalized RADAR BEV features, which includes rescaling the first RADAR BEV features using a first attention mechanism based on the image BEV features and the first RADAR BEV features;
• and
• determining fused data that combines the normalized RADAR BEV features and the image BEV features.

A non-transitory computer-readable medium storing instructions that, when executed by a processing system that includes one or more processors, cause the processing system to perform operations comprising:

• receiving image Bird's Eye View (BEV) features;
• receiving first radio detection and ranging (RADAR) BEV features that are determined based on first RADAR data associated with a first data type;
• determining normalized RADAR BEV features, which includes rescaling the first RADAR BEV features using a first attention mechanism based on the image BEV features and the first RADAR BEV features;
• and
• determining fused data that combines the normalized RADAR BEV features and the image BEV features.

A vehicle, comprising:

• an image sensor;
• a radio detection and ranging (RADAR) system;
• and
• a memory storing processor-readable code;
• and
• at least one processor coupled to the memory, the at least one processor in communication with the image sensor and the RADAR system and configured to execute the processor-readable code to cause the at least one processor to perform operations including:
• receiving image Bird's Eye View (BEV) features that are determined based on image data received from the image sensor;
• receiving first RADAR BEV features that are determined based on first RADAR data associated with a first data type, the first RADAR data received from the RADAR system;
• determining normalized RADAR BEV features, which includes rescaling the first RADAR BEV features using a first attention mechanism based on the image BEV features and the first RADAR BEV features;
• and
• determining fused data that combines the normalized RADAR BEV features and the image BEV features.