System And Method for Edge-Based Multimodal Data Processing and Control Using a Lightweight Language Model and Agent-Orchestration

A system for edge-based multimodal data processing and control, comprising:

• a data collection subsystem configured to acquire sensor data from a plurality of sources, the data collection subsystem operable over multiple communication protocols including at least one of BLE Mesh, Thread, Wi-Fi, and Power Line Communication;
• a data processing layer coupled to the data collection subsystem, the data processing layer including a protocol parsing engine configured to convert the sensor data into a uniform format, a lightweight database or message queue for buffering and storing parsed data, and a set of AI-based tools for specialized analysis of the data, including at least one of computer vision, optical character recognition (OCR), or predictive maintenance;
• a lightweight language model (LLM) stored and executed on an edge device, configured for natural language understanding, text generation, and multimodal reasoning in near-offline or offline mode;
• an agent software module operably connected to the data processing layer and the LLM, the agent configured to orchestrate workflow tasks by invoking various AI-based tools, the LLM, or both, based on scenario triggers or user requests;
• an industrial control interface communicatively linked to one or more control devices selected from the group consisting of PLCs, SCADA systems, or relays, the industrial control interface generating commands to actuators based on outputs from the LLM or the AI-based tools;
• and
• a human-machine interface (HMI) for displaying alarms, diagnostic outputs, and control options.