System and method for multi-modal hyperspectral image generation with cross-modal attention and adaptive quality assurance

A computer system for hyperspectral image generation with multi-modal quality assurance, comprising:

• a hardware memory, wherein the computer system is configured to execute software instructions on nontransitory machine-readable storage media that:
• identify a plurality of spectral bands in a training hyperspectral image;
• compute a correlation coefficient of each spectral band of the plurality of spectral bands to at least one other spectral band of the plurality of spectral bands;
• and
• form a plurality of spectral domain groups based on the computed correlation coefficients;
• obtain multi-modal sensor data comprising RGB, LiDAR, thermal, and NIR inputs;
• perform cross-modal attention calculations between the multi-modal sensor data to determine feature relationships;
• provide the multi-modal sensor data and a plurality of spectral domain groups to a first neural network, wherein the first neural network includes at least one convolutional block, at least one residual block, and cross-modal attention mechanisms;
• obtain as an output of the first neural network, a reconstructed hyperspectral image, based on the multi-modal sensor data;
• and
• detect missing or corrupted data regions in the multi-modal sensor data and apply compensation strategies using information from other modalities;
• analyze a spectral consistency of the reconstructed hyperspectral image;
• perform cross-modal consistency evaluation between different sensor modalities;
• evaluate a RGB reconstruction accuracy between original and reconstructed RGB images;
• detect fusion artifacts specific to multi-modal processing;
• analyze a plurality of noise characteristics in the reconstructed hyperspectral image and the reconstructed RGB image;
• calculate reference-based and reference-free quality metrics;
• generate a plurality of quality scores based on the spectral consistency, cross-modal consistency, RGB reconstruction accuracy, and noise characteristics;
• compare the plurality of quality scores against a predetermined quality threshold;
• and
• update the first neural network based on the quality score comparisons.