Method and system for object-oriented motion-based video description

A method for automatically detecting elementary motion units in order to describe the content of a video sequence, said method comprising the steps of:

• (a) identifying an object of interest and detecting an object-based segment corresponding to the object;
• (b) estimating the dominant motion of the object in frames of the video sequence;
• (c) computing a dissimilarity value representing dominant motion of the object between frames of the video sequence;
• and
• (d) establishing elementary motion units corresponding to significant changes in the dissimilarity value.

A method for automatically detecting elementary motion units in order to describe the content of a video sequence, said method comprising the steps of:

• (a) identifying an object of interest and detecting an object-based segment corresponding to the object;
• (b) providing a dense motion field in each frame representing motion of the object between frames within the object-based segment;
• (c) estimating an affine parameter set from the dense motion field in order to represent the estimated dominant motions of the object;
• (d) computing a dissimilarity value representing dominant motions of the object between adjacent frames;
• (e) identifying local maxima in the dissimilarity value;
• and establishing elementary motion units corresponding to significant changes in the local maxima.

A method for automatically detecting elementary reaction units in order to describe the content of a video sequence, said method comprising the steps of:

• (a) identifying a primary object of interest and an interacting object of interest, and detecting object-based segments corresponding to both objects;
• (b) establishing the spatio-temporal relationship between the primary object and the interacting object from any one or more of the group including (i) coexistence wherein the two objects appear together without any other type of spatio-temporal relationship between them, (ii) spatial contact wherein the boundaries of the two objects overlap and (iii) occlusion wherein one object occludes the other object;
• and
• (c) establishing elementary reaction units descriptive of the spatio-temporal relationship established in step (b).

A method for automatically detecting elementary reaction units in order to describe the content of a video sequence, said method comprising the steps of:

• (a) identifying a primary object of interest and an interacting object of interest, and detecting object-based segments corresponding to both objects;
• (b) identifying overlapping regions of the primary and interacting objects and partitioning the regions into elementary reaction units where the relative motion either remains unchanged between frames or coherent without abrupt and dramatic changes;
• (c) computing a parametric motion model for the motion of the interacting object with reference to the primary object;
• (d) computing the motion trajectory of the interacting object with reference to the primary object;
• and
• (e) providing description of the elementary reaction units in terms of the parametric motion model and the motion trajectory established in steps (c) and (d).