Scalable Visual Computing System

A front-end device for a visual computing system, wherein the front-end device establishes communication connection with an edge service of the visual computing system, and is configured to collect and analyze scene information, the front-end device comprises:

• an image processing module configured to perform pre-processing, compression, coding, and transcoding of the image data, output a compressed video stream carrying timestamp information to the device control and data interaction module, and to output the pre-processed image data to the intelligent computing module and receive a control command sent by the device control and data interaction module to complete the configuration of processing parameters;
• an intelligent computing module configured to perform structured analysis, feature extraction and feature coding of the image data, and output a compact feature stream and a structured result stream to the device control and data interaction module, wherein the intelligent computing module is further configured to receive control commands and receive a model stream and dynamically update an algorithm model, the control commands comprises a parameter configuration command and a function definition command;
• a device control and data interaction module configured to package and encapsulate the received temporal information, spatial information, compressed video stream, pictures, compact feature stream, and structured result stream, transmit them to the edge service, receive, and parse the model stream and control commands issued by the edge service or the cloud service, and transmit the model stream and the control commands to a corresponding processing module, and further to complete the work comprising at least one of process control, device control, status monitoring, model update or transmission control of the front-end device, and acquire operating status and identification information of the device.

An edge service for a visual computing system, wherein the edge service establishes communication connection with a front-end device and a cloud service of the visual computing system, respectively, for convergence, storage, and forwarding of video data and feature data at a local scale, and for resource scheduling and collaborative computing, the edge service comprising:

• an integrated control module configured to receive reported data or commands from the front-end device, control their response process, push a compact feature stream and a structured result stream to the cloud service, receive and forward the control commands or model stream issued by the cloud service, manage an access process and status of the front-end device, listen to the status of the front-end device, and schedule a collaborative operation mode between a plurality of front-end devices;
• a streaming media module configured to receive the compressed video stream, and transcode, capture and package the compressed video stream;
• a calculation processing module configured to complete the work comprising at least one of scheme generation of a multi-node linkage system, image data processing and analysis, multi-node data collaborative analysis, and joint optimization according to the defined functions and tasks;
• and
• a data storage module configured to receive the compressed video stream of the streaming media module, and the compact feature stream and the structured result stream reported by the front-end device, store and manage the compressed video stream, the compact feature stream and the structured result stream in categories, receive a video/picture data fetching command issued by the cloud service, retrieve the compressed video stream or pictures according to conditions, and return the compressed video stream or pictures retrieved to the cloud service.

A cloud service for a visual computing system, wherein the cloud service establishes communication connection with an edge service of the visual computing system, for data converging, collaborative computing, analysis, mining, and decision-making at a global scale, the cloud service comprising:

• a central control module configured to perform configuration management and resource scheduling on all nodes in the system, manage transmission processes of a data stream and a control stream in a unified manner, issue device control, function definition and model update commands to the front-end device, issue a task to the edge service, and receive and process a data reporting command and a status reporting command of the edge service;
• a data center module configured to fetch a compressed video stream or pictures from the edge service on demand, to store, retrieve, and fetch a compact feature stream, a structured result stream, and the compressed video stream or pictures obtained on demand;
• a data analysis module configured to receive the compact feature stream and the structured result stream, or retrieve and fetch data from the data center module according to user commands, converge big data information, analyze and mine the big data information, and extract high-level semantic information and return to a user;
• a calculation simulation module configured to perform structured analysis and processing, simulation prediction, model training, model joint optimization, and collaborative strategy generation, and output computation results;
• an algorithm model repository configured for storage, query and issuance processes and life cycle management of the algorithm model;
• and
• a user interaction module configured to receive a user-related command and return a processing result.

A visual computing method, comprising:

• outputting, by a front-end device, compressed video data and feature data;
• storing the video data, converging the feature data to a cloud service, and transmitting various types of data and control commands, by an edge service;
• and
• storing algorithm models used to support various applications, and returning a corresponding model stream according to a model query command, by the cloud service, realizing a data transmission architecture with multiple streams of video stream, feature stream, and model stream in parallel, and a system architecture of end, edge, and cloud collaboration.