System and method for processing a video stream to extract highlights

A system for extracting video highlights from a video stream comprising:

• storage, containing said video stream in digital form;
• and
• a processor, in communication with said storage, said processor executing software for performing the following functions:
• i. reading an initial plurality of frames from said video stream and constructing a new dictionary based on said initial plurality of frames, said dictionary containing data summarizing the content of said initial plurality of frames;
• ii. breaking the remainder of said video stream into a plurality of segments of approximately equal length;
• iii. reading the next segment from said video stream;
• iv. detecting interest points in said segment and defining a local spatio-temporal cuboid for each of said detected interest points;
• v. calculating, for each of said spatio-temporal cuboids, a plurality of feature vectors and concatenating said plurality of feature vectors into a feature representation for each of said spatio-temporal cuboids;
• vi. determining if a group of feature representations representing all of said spatio-temporal cuboids in said segment can be sparsely reconstructed for a cost not exceeding a predetermined threshold, using existing entries in said dictionary, and, if not, adding said group of feature representations to said dictionary and adding said segment to said video highlights;
• vii. repeating steps iii.-vi. until all segments in said video stream have been processed.