System and methods for real-time class-based inference and adaptive system control

An electronic device, comprising:

• an image sensor configured to capture video frames representing an environment;
• one or more processors;
• and
• memory storing instructions that, when executed by the one or more processors, cause the electronic device to:
• obtain a configuration that defines a plurality of regions of interest each having a polygonal boundary and one or more model-defined object classes of interest;
• preprocess the video frames to normalize one or more input characteristics;
• execute at least one object detection model to produce detections each comprising a location and a class label;
• for each region of interest, determine an in-bound subset of detections based on class membership and spatial inclusion within the polygonal boundary;
• associate in-bound detections across successive frames to maintain track identities over a temporal window without storing raw video frames;
• generate separate region-specific inference outputs for the plurality of regions of interest;
• and
• emit a real-time inference signal to a downstream system based on at least one of the region-specific inference outputs.

A computer-implemented system for monitoring and analyzing congestion trends, comprising:

• one or more video input interfaces configured to receive video data from at least one of a live video stream, a camera-based video source, and a recorded video source;
• an edge device comprising at least one processor and at least one memory storing instructions that, when executed, cause the at least one processor to perform frame-level inference on the video data;
• a processing pipeline executed by the edge device, the processing pipeline comprising a preprocessing module and a normalization module configured to normalize each frame of the video data;
• an object detection module configured to detect objects of interest within each normalized frame using one or more machine learning models;
• a class and region filtering module configured to filter the detected objects based on object class and one or more defined regions of interest;
• a motion vector and feature association module configured to associate the detected objects across frames based on motion and feature data;
• a multi-frame tracking module configured to generate object tracks over multiple frames;
• a multi-frame inference data store configured to store structured inference results generated by the processing pipeline;
• and
• a data ingestion service communicatively coupled to the multi-frame inference data store and configured to receive the structured inference results, the data ingestion service comprising:
• an aggregation module configured to aggregate tracking and detection data across time and across the one or more defined regions of interest;
• an anomaly filtering module configured to detect and filter anomalous data segments;
• and
• a reporting and prediction module configured to generate congestion-related metrics and predictive outputs;
• and
• one or more real-time content consumer systems configured to receive real-time inference outputs from the edge device.

A computer-implemented method for congestion monitoring, comprising:

• receiving video data from at least one video source;
• performing, at an edge device, preprocessing and normalization on frames of the video data;
• detecting objects of interest in the frames using one or more machine learning models;
• filtering the detected objects based on object class and one or more regions of interest;
• associating the detected objects across frames using motion and feature data;
• generating multi-frame object tracks;
• storing structured inference results in a memory;
• transmitting the structured inference results to a data ingestion service;
• aggregating the structured inference results to generate congestion metrics;
• filtering anomalous data segments;
• and
• providing congestion-related outputs to one or more downstream systems.

An edge processing device for video analytics, comprising:

• an image capture module configured to generate video frames;
• at least one processor;
• at least one memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform preprocessing, object detection, region filtering, feature association, and multi-frame tracking;
• and
• a communication interface configured to transmit structured inference artifacts to a remote data ingestion service.