Semantic representation module of a machine-learning engine in a video analysis system

A non-transitory computer-readable medium containing a program, which, when executed on a processor is configured to perform an operation for processing data related to a scene depicted in a sequence of video frames, comprising:

• receiving input data describing an object detected in the scene, wherein the input data comprises at least a classification for the object;
• identifying one or more primitive events, related to a behavior engaged in by the object detected in the scene and wherein each primitive event has an assigned primitive event symbol;
• generating, for the object, a primitive event symbol stream;
• forming a first vector representing the object;
• and
• comparing the first vector to a stored vector to define a similarity measure.

A system, comprising:

• a video input source;
• a processor;
• and
• a memory storing computer instructions, which, when executed on the processor configure the processor to:
• input data describing a scene depicted in a sequence of video frames, the input data comprises at least a classification for an object in a scene;
• identify one or more primitive events, wherein each primitive event provides a semantic value describing a behavior engaged in by the object depicted in the scene and wherein each primitive event has an assigned primitive event symbol;
• generate, for the object, a primitive event symbol;
• form a first vector representing the object;
• and
• compare the first vector to a stored vector to define a similarity measure.