Method for detection of temporal pattern anomalies in video streams

A system for detecting temporal pattern anomalies in a video stream, the system comprising:

• an image input module configured to:
• receive the video stream from a camera;
• an object detection module configured to:
• detect an object in a current frame of the video stream;
• generate a processed current frame that contains the detected object;
• and
• determine a primary class of the detected object;
• a convolutional encoder configured to:
• generate a feature representation of the processed current frame;
• a primary clustering module configured to:
• cluster the feature representation in one or more primary clusters in a clustering space of the primary class, wherein the one or more primary clusters correspond to a sub-class of the primary class;
• a vector generation module configured to:
• generate an information vector of the feature representation, that includes information regarding the primary class, the sub-class of the primary class, and one or more external factors associated with the feature representation;
• and
• an anomaly clustering module configured to:
• cluster each information vector of the video stream into one or more secondary clusters wherein each variable in the information vector is a clustering dimension;
• and
• report a next frame as an anomaly when a corresponding information vector is positioned outside a secondary cluster of the one or more secondary clusters of an information vector of a previous frame.

A computer programmable product for detecting temporal pattern anomalies in a video stream, the computer programmable product comprising a set of instructions, the set of instructions when executed by a processor causes the processor to:

• receive the video stream from a camera;
• detect an object in a current frame of the video stream and generate a processed current frame that contains the detected object;
• determine a primary class of the detected object;
• generate a feature representation of the processed current frame by a convolutional encoder;
• cluster the feature representation m one or more primary clusters in a clustering space of the primary class, wherein the one or more primary clusters correspond to a sub-classes of the primary class;
• generate an information vector of the feature representation, that includes information regarding the primary class, the sub-class of the primary class, and one or more external factors associated with the feature representation;
• cluster each information vector of the video stream into one or more secondary clusters, wherein each variable in the information vector is a clustering dimension;
• and
• report a next frame as an anomaly when a corresponding information vector is positioned outside a secondary cluster, of the one or more secondary clusters of an information vector of a previous frame.