Time-constrained keyframe selection method

Abstract

A method for candidate frame selection involves sampling the source frames of the source video at a predetermined fixed periodic interval. The largest frame differences represent candidate boundaries, and the frames before and after the N/2 largest candidate boundaries are selected as candidate frames. A method for selecting keyframes involves clustering all candidate frames into a hierarchical binary tree using a hierarchical agglomerative clustering algorithm. Optionally, the pairwise distances for the members of the two clusters is modified according to class membership of the members, which is preferably determined statistically from image class statistical models. A method for selecting M clusters from which keyframes are extracted involves splitting the M-1 largest clusters of a hierarchical binary tree of clusters. Optionally, clusters not having at least one uninterrupted sequence of frames of at least a minimum duration are filtered out. The source video duration is divided into equal duration intervals. If an interval has no keyframes, all other intervals having at least two keyframes are inspected in descending keyframe count order to attempt to find a keyframe within a cluster that has a member within the interval that does not have any keyframes. If such a keyframe is found, the member is substituted as the keyframe for the cluster. The minimum time between any two keyframes is determined. If this minimum time is less than a minimum time threshold, an attempt is made to find another keyframe from one or both of the two clusters which the two conflicting keyframe belong to. If a substitute cannot be found, one of the conflicting keyframes is deleted.

Dependencies are inferred from the claim text.