Method, device, and storage medium for deep learning based domain adaptation with data fusion for aerial image data analysis

A domain adaptation for efficient learning fusion (DAELF) device, comprising:

• a memory, configured to store program instructions for performing a DAELF method;
• and
• a processor, coupled with the memory and, when executing the program instructions, configured for:
• acquiring data from a plurality of data sources of a plurality of sensors;
• for each of the plurality of sensors, training an auxiliary classifier generative adversarial network (AC-GAN) by a hardware processor, wherein the AC-GAN includes a feature extraction network, a label prediction network, a generator network, and a discriminator network, with data from each data source of the plurality of data sources, thereby obtaining a trained feature extraction network and a trained label prediction network for each data source;
• using the trained feature extraction network and the trained label prediction network for each data source on a sensor side, and a corresponding centralized fusion network on a fusion center side to form a decision-level fusion network;
• or using the trained feature extraction network for each data source on the sensor side and a corresponding centralized fusion network on the fusion center side to form a feature-level fusion network;
• training the decision-level fusion network or the feature-level fusion network with a source-only mode or a generate to adapt (GTA) mode;
• and
• applying the trained decision-level fusion network or the trained feature-level fusion network to detect a target of interest.

A non-transitory computer-readable storage medium, containing program instructions for, when being executed by a processor, performing a domain adaptation for efficient learning fusion (DAELF) method, the method comprising:

• acquiring data from a plurality of data sources of a plurality of sensors;
• for each of the plurality of sensors, training an auxiliary classifier generative adversarial network (AC-GAN) by a hardware processor, wherein the AC-GAN includes a feature extraction network, a label prediction network, a generator network, and a discriminator network, with data from each data source of the plurality of data sources, thereby obtaining a trained feature extraction network and a trained label prediction network for each data source;
• using the trained feature extraction network and the trained label prediction network for each data source on a sensor side, and a corresponding centralized fusion network on a fusion center side to form a decision-level fusion network;
• or using the trained feature extraction network for each data source on the sensor side and a corresponding centralized fusion network on the fusion center side to form a feature-level fusion network;
• training the decision-level fusion network or the feature-level fusion network with a source-only mode or a generate to adapt (GTA) mode;
• and
• applying the trained decision-level fusion network or the trained feature-level fusion network to detect a target of interest.