Multi-modal large model construction and operation method and system for reservoir dam safety

A multi-modal large model construction and operation method for reservoir dam safety, including the following steps:

• constructing a training set:
• collecting historical multi-modal monitoring data, external resource data, historical dam data, hidden danger locations and types, hidden danger propagation paths, forming input samples based on said data;
• calculation data for training labels originates from historical dam data, hidden danger locations and types, and hidden danger propagation paths;
• the process of obtaining the hidden danger propagation path includes:
• defining each structural unit of the dam as a structural unit node and assigning attributes;
• defining physical connections and mechanical relationships between structural units as structural connection edges and assigning attributes;
• establishing association relationships between nodes and edges:
• establishing edges between structural unit nodes based on historical operational state data, engineering technical documents, and geometric information from the BIM model;
• storing node and edge information in a graph database;
• performing calculations on the nodes and edges in the graph database to obtain node connectivity and importance scores;
• updating the health status and risk level of nodes in the knowledge graph, as well as the weights and influence probability of edges, based on real-time collected dam operational state data and new external information resources;
• analyzing the path and probability of a hidden danger propagating from one structural unit to another using the updated node and edge attributes and the results of connectivity calculations;
• based on the hidden danger propagation probability, constructing a matrix representing the hidden danger correlation between nodes;
• calculating the shortest path of hidden danger propagation from a source node to other nodes by combining the hidden danger influence matrix and the graph database;
• identifying nodes that have significant influence in the hidden danger propagation process based on their importance score and role in the propagation process;
• the individual samples in the training set include:
• as model input multi-modal data:
• sensor data deployed on the dam, historical hidden danger records, engineering technical documents formed during various stages of dam planning, design, construction, and operation, dam safety-related information resources, geometric and material information of the BIM model, timestamp and spatial location information;
• and as training labels, whether a hidden danger event occurred and the probability value of the hidden danger event occurring;
• training a multi-modal prediction model based on the Transformer architecture using the training set;
• performing real-time prediction and early warning for dam safety hazards using the trained model, based on real-time collected monitoring data, the type and location of hidden dangers, historical dam data related to the hidden dangers, the propagation path of hidden dangers within the dam, and external resource data related to the hidden dangers;
• calculating the actual impact degree of the hidden danger event, a comprehensive risk score, warning thresholds for sensors associated with the hidden danger event, and emergency measure suggestions based on the probability value of whether the hidden danger event occurred output by the model.