US 12,755,528 B1Granted

System and method for a multi-scale adaptive scale-space pyramid network for aerial scenes recognition

AssigneePrince Mohammad Bin Fahd University

InventorsMuhammad Attique Khan, Ghassen Ben Brahim

Life of this patent

Estimating the term…

Loading patent…

Abstract

A remote sensing system for performing a method of real-time interpretation of aerial images includes remote imaging platforms capturing Earth images at varying scales. A machine learning processing circuitry employs a deep learning model with a Gaussian pyramid module to generate multi-scale feature maps. A scale selection block upsamples coarser maps to the finest scale for uniformity. Separate feature extractors convert these maps into vectors, which are combined via a scale attention module to compute attention weights. A cross-propagation feature module utilizes an uncertainty network to generate pixel-wise uncertainty maps, thereby guiding feature fusion into enhanced scale feature maps. A complexity-aware pooling mechanism preserves semantic features, which are then fed to a classification head that assigns class labels. The system outputs accurately labeled aerial images, enhancing robustness and precision in multi-scale aerial image recognition.

Claims20

Dependent claims

A remote sensing system for real time interpretation of aerial images, comprising:

a plurality of remote imaging platforms for capturing a plurality of aerial images of the Earth at multiple resolutions;
machine learning processing circuitry configured for a deep learning model including an input device configured to input an image of the plurality of aerial images;
a Gaussian pyramid module configured to learn to generate multiple feature maps, at different original scales ranging from coarse to fine scales, from the input image;
a scale selection block to recalibrate significance of features in the feature maps across the different original scales dynamically, by upsampling a coarsest three of the feature maps to match dimensions of the finest feature map, such that the multiple feature maps are at a same dimension;
a feature extraction module having a plurality of separate feature extractors to convert respective feature maps from the scale selection block into 1D vectors, wherein the 1D vectors are stacked across scales to form a single vector, which is passed through a scale attention module to obtain attention weights;
a cross propagation feature module that passes the generated multiple feature maps from the Gaussian pyramid module through an uncertainty network to generate a corresponding uncertainty map, which contains an uncertainty score for each pixel in the input image, wherein the generated multiple feature maps are concatenated with each other according to the uncertainty maps to generate enhanced feature maps, which undergo fusion and addition to generate an enhanced scale feature map;
a complexity-aware pooling mechanism to preserve both semantic and discriminative features based on the recalibrated feature maps and the enhanced feature maps;
and a classification head to classify the aerial images into a corresponding class;
and an output device configured to output the aerial images and a respective label for the corresponding class.

6 more elements

A method for real time interpretation of aerial images by a deep learning model, comprising:

capturing, by a plurality of remote imaging platforms, a plurality of aerial images of the Earth at multiple resolutions;
inputting, by an input device, an image of the plurality of aerial images;
learning to generate, by a Gaussian pyramid module of machine learning circuitry, multiple feature maps, at different original scales ranging from coarse to fine scales, from the input image;
recalibrating significance, by a scale selection block, of features in the feature maps across the different original scales dynamically, by upsampling a coarsest three of the feature maps to match dimensions of the finest feature map, such that the multiple feature maps are at a same dimension;
converting, by a feature extraction module having a plurality of separate feature extractors, respective feature maps from the scale selection block into 1D vectors, wherein the 1D vectors are stacked across scales to form a single vector, which is passed through an attention module to obtain attention weights;
passing, by a cross propagation feature module, the generated multiple feature maps from the Gaussian pyramid module through an uncertainty network to generate a corresponding uncertainty map, which contains an uncertainty score for each pixel in the input image, wherein the generated multiple feature maps are concatenated with each other according to the uncertainty maps to generate enhanced feature maps, which undergo fusion and addition to generate an enhanced scale;
preserving, by a complexity-aware pooling mechanism, both semantic and discriminative features based on the recalibrated feature maps and the enhanced feature maps;
classifying, by a classification head, the aerial images into a corresponding class;
and outputting, by an output device, the aerial images and a respective label for the corresponding class.

6 more elements

Dependencies are inferred from the claim text.