Spatio-activity based mode matching

An apparatus, in relation to a video frame comprising a visual element having visual characteristics and corresponding to a location in the current video frame, for matching at least one of a plurality of mode models to the visual element, comprising:

• a memory and a processor cooperating to function as:
• an associating unit configured to associate the visual element of the current video frame with the plurality of mode models, each corresponding to the same location as the visual element and having visual and temporal characteristics;
• and
• a matching unit configured to match at least one of the plurality of mode models to the visual element of the current video frame, said matching unit further comprising:
• a first determining unit configured to determine a visual support value for each mode model depending upon a similarity between the visual characteristics of the visual element and of the mode model;
• a second determining unit configured to determine a spatial support value for each mode model depending upon a similarity between the temporal characteristics of the mode model and of mode models associated with one or more other visual elements in the vicinity of the visual element of the current video frame;
• and
• an identifying unit configured to identify a matching one of the plurality of mode models depending upon the visual support values and the spatial support values.

A non-transitory computer-readable storage medium having recorded thereon a computer program for directing a processor to execute a method, in relation to a video frame comprising a visual element having visual characteristics and corresponding to a location in the current video frame, of matching at least one of a plurality of mode models to the visual element, comprising:

• associating the visual element of the current video frame with the plurality of mode models, each corresponding to the same location as the visual element and having visual and temporal characteristics;
• and
• matching at least one of the plurality of mode models to the visual element of the current video frame, said matching step further comprising:
• determining a visual support value for each mode model depending upon a similarity between the visual characteristics of the visual element and of the mode model;
• determining a spatial support value for each mode model depending upon a similarity between the temporal characteristics of the mode model and of mode models associated with one or more other visual elements in the vicinity of the visual element of the current video frame;
• and
• identifying a matching one of the plurality of mode models depending upon the visual support values and the spatial support values.

A method, in relation to a current video frame comprising a visual element having visual characteristics and corresponding to a location in the current video frame, of matching the visual element to at least one of a plurality of mode models in a visual element model, comprising the steps of:

• associating said visual element of the current video frame with the plurality of mode models, each corresponding to the same location as the visual element and having visual and temporal characteristics;
• and
• matching at least one of the plurality of mode models to the visual element of the current video frame, said matching step further comprising the steps of:
• determining a visual support value for each mode model based on a correspondence between visual characteristics of the visual element and of the mode model in the visual element model;
• determining a spatial support value for each mode model based on a correspondence between temporal characteristics of the mode model and of one or more mode models associated with visual elements at locations other than the location of the visual element in the vicinity of the visual element of the current video frame;
• and
• selecting a matching mode model from the plurality of mode models based at least on the visual support values and the spatial support values.