US 9,805,270 B2 · Video segmentation techniques

Abstract

A video segmentation system can be utilized to automate segmentation of digital video content. Features corresponding to visual, audio, and/or textual content of the video can be extracted from frames of the video. The extracted features of adjacent frames are compared according to a similarity measure to determine boundaries of a first set of shots or video segments distinguished by abrupt transitions. The first set of shots is analyzed according to certain heuristics to recognize a second set of shots distinguished by gradual transitions. Key frames can be extracted from the first and second set of shots, and the key frames can be used by the video segmentation system to group the first and second set of shots by scene. Additional processing can be performed to associate metadata, such as names of actors or titles of songs, with the detected scenes.

Claims20

Dependent claims

A computer-implemented method, comprising:

determining a feature for a frame of a plurality of frames of a video;
analyzing a similarity between the feature and at least one feature associated with adjacent frames to the frame to determine a first shot of the video;
determining that the first shot meets a time threshold;
determining that a similarity metric between a first frame of the first shot and a second frame of the first shot meets a dissimilarity threshold;
determining that a similarity matrix of at least a subset of frames of the first shot corresponds to a dissolve pattern, the subset of frames corresponding to at least one second shot of the video;
generating a graph of the video, the graph comprising nodes corresponding to the first shot and the at least one second shot;
and
determining a grouping of the first shot and the at least one second shot by performing one or more cuts of the graphs.

5 more elements

A non-transitory computer-readable storage medium comprising instructions that, upon being executed by a processor of a computing device, cause the computing device to:

determine a feature for a frame of a plurality of frames of a video;
analyze a similarity between the feature and at least one feature associated with adjacent frames to the frame to determine a first shot of the video;
determine that the first shot meets a time threshold;
determine that a similarity metric between a first frame of the first shot and a second frame of the first shot meets a dissimilarity threshold;
determine that a similarity matrix of at least a subset of frames of the first shot corresponds to a dissolve pattern, the subset of frames corresponding to at least one second shot of the video;
generate a graph of the video, the graph comprising nodes corresponding to the first shot and the at least one second shot;
and
determine a grouping of the first shot and the at least one second shot by performing one or more cuts of the graphs.

5 more elements

A computing device, comprising:

a processor;
memory including instructions that, upon being executed by the processor, cause the computing device to:
determine a feature for a frame of a plurality of frames of a video;
analyze a similarity between the feature and at least one feature associated with adjacent frames to the frame to determine a first shot of the video;
determine that the first shot meets a time threshold;
determine that a similarity metric between a first frame of the first shot and a second frame of the first shot meets a dissimilarity threshold;
determine that a similarity matrix of at least a subset of frames of the first shot corresponds to a dissolve pattern, the subset of frames corresponding to at least one second shot of the video;
generate a graph of the video, the graph comprising nodes corresponding to the first shot and the at least one second shot;
and
determine a grouping of the first shot and the at least one second shot by performing one or more cuts of the graphs.

7 more elements

Dependencies are inferred from the claim text.

Video segmentation techniques

Life of this patent

Video segmentation techniques

Life of this patent

Abstract

Claims20