Method and system for video summarization
AssigneeFuturewei Technologies, Inc.
InventorsJizhou Gao, Yu Huang, Hong Heather Yu
Life of this patent
Estimating the term…
Abstract
A video summary method comprises dividing a video into a plurality of video shots, analyzing each frame in a video shot from the plurality of video shots, determining a saliency of each frame of the video shot, determining a key frame of the video shot based on the saliency of each frame of the video shot, extracting visual features from the key frame and performing shot clustering of the plurality of video shots to determine concept patterns based on the visual features. The method further comprises fusing different concept patterns using a saliency tuning method and generating a summary of the video based upon a global optimization method.
Claims20
A method comprising:
- dividing a video into a plurality of video shots;
- analyzing each frame in a video shot from the plurality of video shots;
- determining a saliency of each frame of the video shot;
- determining a key frame of the video shot based on the saliency of each frame of the video shot;
- extracting visual features from the key frame;
- performing shot clustering of the plurality of video shots to determine concept patterns based on the visual features;
- fusing different concept patterns using a saliency tuning method, wherein a saliency value obtained in the saliency tuning method is generated by fusing a visual saliency value, an audio saliency value, a motion saliency value and a spatial color saliency value;
- and
- generating a summary of the video based upon a global optimization method.
A system comprising:
- an extracting unit extracting a plurality of video shots from a video;
- an analyzer determining a saliency of each frame of a video shot from the plurality of video shots;
- a key frame determiner determining a key frame of the video shot based on the saliency of each frame in the video shot;
- a visual feature extractor extracting visual features from the key frame;
- a shot clustering block performing shot clustering of the plurality of video shots to determine concept patterns based on the visual features;
- a saliency tuning block performing fusing different concept patterns using a saliency tuning method, wherein a saliency value obtained in the saliency tuning method is generated by fusing a visual saliency value, an audio saliency value, a motion saliency value and a spatial color saliency value;
- and
- a summary generator generating a video summary based upon a global optimization method, wherein the global optimization method is carried out in a plurality of limited ranges, and wherein the plurality of limited ranges include a first range comprising a starting shot and a second range comprising an ending shot.
A non-transitory computer readable medium with an executable program stored thereon, wherein the program instructs a microprocessor to perform the following steps:
- analyzing each frame in a video shot from a plurality of video shots, analyzing comprising determining a saliency of each frame of the video shot;
- determining a key frame of the video shot based on the saliency of each frame the video shot;
- extracting visual features from the key frame;
- performing shot clustering of the plurality of video shots to determine concept patterns based on the visual features;
- fusing different concept patterns using a saliency tuning method, wherein a saliency value obtained in the saliency tuning method is generated by fusing a visual saliency value, an audio saliency value, a motion saliency value and a spatial color saliency value;
- and
- generate a video summary based upon a global optimization method.
Dependencies are inferred from the claim text.