Railway anomaly detection method and system based on multi-modal data fusion

Abstract

This disclosure relates to a railway anomaly detection method and a system thereof based on multi-modal data fusion, comprising the following steps: encoding respective modal data acquired from railway environments, and concatenating the encoded multi-modal data features; wherein the modal data comprises 1D vibration signals, 2D image data, and 3D point cloud information; performing automatic weight classification on the concatenated multi-modal data features through an attention mechanism to obtain weighted feature vectors fused with multi-modal information; inputting the feature vectors with positional encoding into a Segment Anything Model (SAM) encoder to generate segmentation results; and determining railway anomalies based on the segmentation results. This disclosure demonstrates high monitoring accuracy, real-time responsiveness, and targeted detection capabilities.

Dependencies are inferred from the claim text.