Automatic tracking of people and bodies in video

A computer program product for tracking faces and bodies in a video comprising a computer-readable storage medium storing program code for executing the following steps:

• receiving a video for analysis, the video comprising a plurality of frames;
• calculating a difference in intensity between different regions of any face in the frames;
• generating a plurality of face detections for each face in the frames;
• dividing each color in each frame into a plurality of bins;
• generating a histogram for each frame based on the bins;
• smoothing each histogram;
• concatenating the smoothed histograms;
• predicting a location of a face in each frame;
• selecting a face detection from the plurality of face detections for each face in the frame that is closest to the predicted location of the face;
• selecting a reference position for the face detection on a first histogram at time t;
• comparing the reference position for the first histogram to a reference position for a face detection of a second histogram at time t+1;
• calculating a distance between the reference positions for each subsequent histogram in preparation for creating a face track from the face detection;
• comparing each histogram with a subsequent consecutive histogram to determine whether a difference in a number of bins of color for each histogram exceed a predefined threshold;
• defining the exceeded difference as a shot boundary;
• detecting all shot boundaries in the video;
• normalizing and rectifying faces in each frame to align a plurality of features in the face;
• calculating a distance between the normalized and rectified faces in the frames;
• generating a similarity matrix between tracks based on the distance between tracks;
• clustering tracks to group together face tracks for a each person in the video;
• and
• attaching a body outline to each frame within a face track.

A system for tracking faces and bodies in videos, comprising:

• a memory;
• a processor, the processor configured to implement instructions stored in the memory, the memory storing executable instructions and a video for analysis, the video comprising a plurality of frames;
• a facial detection module for calculating a difference in intensity between a plurality of regions of any face that is present in a frame and generating a plurality of face detections, the facial detection module generating a histogram for each frame;
• a filter for smoothing the histogram for each frame and concatenating the smoothed histograms;
• a tracking module for predicting a location of a face in each frame, comparing the location to the plurality of face detections for each face, and selecting the face track that is closest to the predicted location as long as the overlap between the predicted location and the selected face track exceeds a threshold level, the tracking module detecting all shot boundaries in the video by calculating a distance between the reference positions for each subsequent histogram and comparing each histogram with a subsequent consecutive histogram to determine whether a difference in color for each histogram exceeds a predefined threshold;
• a clustering module for normalizing and rectifying the faces in the frames and clustering the normalized and rectified faces for each person in the video;
• and
• a body detection module for generating a body outline and associating it with the face diction.