Object tracking for autonomous vehicles using long-range acoustic beamforming combined with RGB visual data

A computer-implemented method, comprising:

• generating spatial beamforming maps locating a sound source based upon acoustic signals received at a plurality of acoustic sensors of a network of sensors;
• identifying a type of an object generating the acoustic signals received at the plurality of acoustic sensors based upon comparison of the acoustic signals with a plurality of acoustic signals and respective objects stored in a dataset, wherein the dataset is a multimodal long-range beamforming dataset including audio visual data corresponding to a plurality of objects;
• and
• generating feature maps for an application in an autonomous vehicle driving by enhancing visualization maps generated based upon visual signals received by a plurality of visual sensors of the network of sensors.

A non-transitory computer-readable medium (CRM) embodying programmed instructions which, when executed by at least one processor of an autonomous vehicle, cause the at least one processor to perform operations comprising:

• generating spatial beamforming maps locating a sound source based upon acoustic signals received at a plurality of acoustic sensors of a network of sensors;
• identifying a type of an object generating the acoustic signals received at the plurality of acoustic sensors based upon comparison of the acoustic signals with a plurality of acoustic signals and respective objects stored in a dataset, wherein the dataset is a multimodal long-range beamforming dataset including audio visual data corresponding to a plurality of objects;
• and
• generating feature maps for an application in an autonomous vehicle driving by enhancing visualization maps generated based upon visual signals received by a plurality of visual sensors of the network of sensors.