Recurrent super-resolution radar for autonomous vehicles

A radar system in an ego vehicle, comprising:

• an antenna module comprising one or more meta-structure antennas that are configured to radiate one or more transmission radio frequency (RF) beams to a target and receive one or more return RF beams reflected from a surrounding environment of the ego vehicle;
• and
• a perception module coupled to the antenna module and configured to detect and identify one or more targets from the one or more return RF beams,
• wherein the perception module comprises one or more deep learning neural networks that are trained with a first set of radar data with a first resolution and a set of lidar data with a second resolution greater than the first resolution and configured to produce a second set of radar data with a resolution that substantially corresponds to the second resolution of the set of lidar data from the first set of radar data with the first resolution,
• wherein each of the one or more deep learning neural networks comprises a plurality of recurrent encoder layers,
• wherein each of the plurality of recurrent encoder layers is configured to perform one or more recurrence operations prior to a max pooling operation,
• wherein the second set of radar data is produced from at least an output of the plurality of recurrent encoder layers,
• wherein the one or more deep learning neural networks are trained with a training dataset that includes one or more image sets of the first set of radar data associated with first timestamp data and the lidar data associated with second timestamp data, and
• wherein the one or more deep learning neural networks are further trained to map the first set of radar data into the second set of radar data that substantially corresponds to a representation of the lidar data that is nearest in time to the first timestamp data.

A method of operating a radar system in an autonomous driving system of an ego vehicle, the method comprising:

• directing a beam steering antenna structure to generate one or more radio frequency (RF) beams and radiate the one or more RF beams to one or more targets in a surrounding environment of the ego vehicle;
• providing a first set of radar data with a first resolution from one or more return RF beams that are reflected from the one or more targets, to a recurrent super-resolution network;
• producing a second set of radar data with a second resolution different from the first set of radar data with the first resolution using one or more trained neural networks in the recurrent super-resolution network,
• wherein the recurrent super-resolution network is configured to map a lower-resolution radar dataset that corresponds to the first set of radar data with the first resolution into a higher-resolution radar dataset that corresponds to the second set of radar data with the second resolution to facilitate identification of the one or more targets,
• wherein each of the one or more trained neural networks comprises a plurality of recurrent encoder layers,
• wherein each of the plurality of recurrent encoder layers is configured to perform one or more recurrence operations prior to a max pooling operation, and
• wherein the second set of radar data with the second resolution is produced from at least an output of the plurality of recurrent encoder layers;
• generating target identification information from the second set of radar data with the second resolution to identify the one or more targets,
• obtaining radar data with associated first timestamp data from one or more radar sensors, wherein the radar data is captured over a continuous field of view;
• obtaining lidar data with associated second timestamp data from one or more lidar sensors;
• associating the radar data with the lidar data that is substantially corresponding in time with the radar data using the first timestamp data and the second timestamp data;
• and
• forming a training dataset that includes the radar data and the lidar data, wherein the radar data has a resolution lower than that of the lidar data.