Layered motion representation and extraction in monocular still camera videos

A system for segmenting motion data and inferring layering order in a video, the system comprising:

• a memory to store one or more instructions;
• and
• a processor communicatively coupled to the memory, and configured to execute the one or more instructions stored in the memory, wherein the processor comprises:
• a video segmentation module for segmenting a first image frame of the video using intra-frame mode estimation, and segmenting one or more further image frames of the video using inter-frame mode matching;
• a label frame difference generation module for generating a label change map for a current segmented image frame by generating a binary map of the current segmented image frame, and then marking one or more pixels whose labels have changed with respect to a previous segmented image frame;
• a non-still segment detection module for detecting one or more non-still image segments of the current segmented image frame based on the label change map;
• a moving segment detection module for detecting one or more moving image segments in the one or more non-still image segments;
• a motion estimation module for estimating motion of the one or more moving image segments;
• and
• a motion segmentation module for representing each moving segment as a vector of seven parameters, and clustering one or more similar moving segment vectors based on corresponding one or more parameters, wherein the seven parameters include two spatial coordinates of a centroid of corresponding segment, four affine parameters including translation and rotation parameter corresponding to a similarity transform, and a weight defined by total number of corresponding pixels.

A non-transitory computer readable medium configured to store a program causing a computer to segment motion data and inferring layering order in a video, said program configured to:

• segment a first image frame of the video using intra-frame mode estimation;
• segment one or more further image frames of the video using inter-frame mode matching;
• generate a label change map for a current segmented image frame by generating a binary map of the current segmented image frame;
• mark one or more pixels whose labels have changed with respect to a previous segmented image frame;
• detect one or more non-still image segments of the current segmented image frame based on the label change map;
• detect one or more moving image segments in the one or more non-still image segments;
• estimate motion of the one or more moving image segments;
• represent each moving segment as a vector of seven parameters, wherein the seven parameters include two spatial coordinates of a centroid of corresponding segment, four affine parameters including translation and rotation parameter corresponding to a similarity transform, and a weight defined by total number of corresponding pixels;
• and
• cluster one or more similar moving segment vectors based on corresponding one or more parameters.

A monocular still camera surveillance system, comprising:

• a monocular still camera for capturing the video;
• a memory to store one or more instructions;
• and
• a processor coupled to the memory and configured to execute the one or more instructions stored in the memory to:
• segment a first image frame of the video using intra-frame mode estimation;
• segment one or more further image frames of the video using inter-frame mode matching;
• generate a label change map for a current segmented image frame by generating a binary map of the current segmented image frame, and then marking one or more pixels whose labels have changed with respect to a previous segmented image frame;
• detect one or more non-still image segments of the current segmented image frame based on the label change map;
• detect one or more moving image segments in the one or more non-still image segments;
• estimate motion of the one or more moving image segments;
• represent each moving segment as a vector of seven parameters, wherein the seven parameters include two spatial coordinates of a centroid of corresponding segment, four affine parameters including translation and rotation parameter corresponding to a similarity transform, and a weight defined by total number of corresponding pixels;
• cluster one or more similar moving segment vectors based on corresponding one or more parameters;
• and
• performing motion detection and activity recognition based on the one or more clusters.