Multimodal sensing, fusion for machine perception

A radar sensor system comprising:

• multiple respective sensor units disposed upon a mobile platform such that the respective sensor units have respective nominal geometric relationships with one another;
• a centralized processor;
• a communication network to communicate information between the multiple respective sensor units;
• wherein respective sensor units include,
• a respective radar receiver to receive respective radar data representing a respective radar scene associated with the respective sensor unit;
• a respective vision sensor that is positioned to capture a respective image within a respective vision sensor field of view that overlaps the respective radar scene and that produces respective vision sensor-based information indicative of the respective captured image;
• a respective local processor operably coupled to a respective local non-transitory computer readable medium, the respective local non-transitory computer readable medium comprising a respective first plurality of instructions stored in association therewith that are accessible to, and executable by, the respective local processor, wherein the respective plurality of instructions includes:
• instructions that, when executed, send at least a respective portion of the respective radar data and the respective vision sensor-based information to the centralized processor;
• wherein the centralized processor is operably coupled to a second non-transitory computer readable medium, the second non-transitory computer readable medium comprising a second plurality of instructions stored in association therewith that are accessible to, and executable by, the centralized processor, wherein the second plurality of instructions includes:
• instructions that, when executed, use the respective vision sensor-based information to determine respective differences between respective sensor nominal locations and respective sensor actual locations and to use the respective determined differences to fuse the respective portions of the radar data received over the network from respective sensor units to produce a wide radar aperture.

A method to produce a wide radar aperture comprising:

• placing multiple respective sensor units that each includes a respective radar receiver and a respective location sensor at respective nominal locations of a mobile platform such that the respective sensor units have a nominal geometric relationship;
• at each of the multiple sensor units, receiving respective radar data at the respective radar receivers at respective sensor units representing respective radar scenes associated with the respective sensor units;
• at each of the multiple sensor units, using respective location sensors at respective sensor units to determine respective actual locations of the respective sensor units;
• determining for one or more of the multiple respective sensor units, a respective location sensor-based difference between the respective sensor unit nominal location and the respective sensor unit actual location;
• fusing respective radar data received at the multiple respective sensor units based at least in part upon the respective nominal geometric relationships and respective location sensor-based differences determined for the one or more of the multiple respective sensor units, between respective sensor unit nominal location and respective sensor actual location, to produce the wide radar aperture.