System and method for shot change detection in a video sequence
AssigneeSpinella IP Holdings, Inc.
InventorsWilliam L. Gaddy, Vidhya Seran
Life of this patent
Estimating the term…
Abstract
A computer implemented method for automatically identifying shot changes in a video sequence in real-time or near-real-time is disclosed. Optical flow energy change differences between frames, sum-of-square differences between optical-flow-compensated frames, and hue histogram changes within frames are analyzed and stored in frame buffers. A feature vector formed from a combination of these measurements is compared to a feature vector formed from thresholds based on tunable recall and precision to declare the presence or absence of a shot change.
Claims27
A method, comprising:
- receiving, using a processor, a plurality of frames;
- determining an image similarity metric based on at least one pair of frames of the plurality of frames, wherein the similarity metric comprises determining an optical flow energy change difference measure (OFEDM) between optical flow fields comprising:
- calculating a first optical flow field between a current frame and a previous frame and a second optical flow field between the previous frame and a previous third frame;
- comparing the first optical flow field to the second optical flow field with respect to a direction and a magnitude of each pixel to arrive at a similarity scalar value for each pixel;
- and
- calculating a mean value of each of the similarity scalar values as the OFEDM;
- and
- detecting a shot change in the plurality of frames upon the mean scalar value exceeding a user-supplied threshold.
An apparatus, comprising:
- at least one processor configured to:
- receive a plurality of frames;
- determine an image similarity metric based on at least one pair of frames of the plurality of frames, wherein the similarity metric comprises determining an optical flow energy change difference measure (OFEDM) between optical flow fields comprising:
- calculating a first optical flow field between a current frame and a previous frame and a second optical flow field between the previous frame and a previous third frame;
- comparing the first optical flow field to the second optical flow field with respect to a direction and a magnitude of each pixel to arrive at a similarity scalar value for each pixel;
- and
- calculating a mean value of each of the similarity scalar values as the OFEDM;
- and
- detect a shot change in the plurality of frames upon the mean scalar value exceeding a user-supplied threshold.
The system of 18, further comprising a plurality of first-in first-out (FIFO) frame buffers for storing the plurality of frames and the at least one of the SSDM, OFEDM, and HHEDM.
A non-transitory computer-readable medium comprising computer-executable instructions to:
- receive a plurality of frames;
- determine an image similarity metric based on at least one pair of frames of the plurality of frames, wherein the similarity metric comprises determining an optical flow energy change difference measure (OFEDM) between optical flow fields comprising:
- calculating a first optical flow field between a current frame and a previous frame and a second optical flow field between the previous frame and a previous third frame;
- comparing the first optical flow field to the second optical flow field with respect to a direction and a magnitude of each pixel to arrive at a similarity scalar value for each pixel;
- and
- calculating a mean value of each of the similarity scalar values as the OFEDM;
- and
- detect a shot change in the plurality of frames upon the mean scalar value exceeding a user-supplied threshold.
Dependencies are inferred from the claim text.