# US 7,612,832 B2 · Method and system for video clip compression > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,612,832 B2 - Document ID: US7612832B2 - Status: Granted - Kind: B2 - Application number: 11/092,389 - Filed: 2005-03-29 - Published: 2006-10-12 - Granted: 2009-11-03 - Earliest claimed: 2005-03-29 - Assignees: Microsoft Corporation - Inventors: Zhengyou Zhang; Zicheng Liu - Web page: https://patentgrep.com/patent/US7612832B2 ## CPC - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/587: involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In a method for compressing a video clip containing audio content and image content, an image and/or an audio portion of individual video frames of the video clip are analyzed. Next frame scores are calculated for the video frames. Each frame score is based on at least one image attribute of the image of the video frame, and/or an audio attribute of the audio portion of the video frame. Next, key frames are identified that have a frame score that exceeds a threshold frame score. Finally, a compressed video clip is formed in which the images of non-key frames are removed. A system for implementing the method is also disclosed. ## Claims ### Claim 1 A method for compressing a video clip, implemented by a computer comprising a tangible storage medium, the method comprising: analyzing at least one of an image and an audio portion of individual video frames of the video clip; calculating frame scores for the individual video frames, each frame score based on at least one of an image attribute of the image of the video frame and an audio attribute of the audio portion of the video frame; identifying key frames of the video frames having a frame score that exceeds a threshold frame score; forming a compressed video clip in which the images of non-key video frames are removed, the compressed video clip comprising the images of the key frames, the audio portions of the key frames and the audio portions of the non-key video frames. ### Claim 2 (depends on 1) The method of claim 1, wherein forming a compressed video clip in which the images of non-key video frames are removed includes synchronizing the images of the identified key frames with the audio portions of the key frames and the non-key video frames. ### Claim 3 (depends on 1) The method of claim 1, wherein: each video frame includes a time code period; and the method further comprises filling in image gaps of the non-key video frames of the compressed video clip with the images of the key frames including extending the time code period for the images of the key frames. ### Claim 4 (depends on 1) The method of claim 1, wherein: each frame score is based on a contrast measurement of the image of the video frame, which indicates a relative intensity difference between neighboring pixels of the image of the video frame; and calculating frame scores for the individual video frames comprises assigning higher frame scores to the video frames whose images have a higher relative intensity difference between neighboring pixels than the video frames whose images have a lower relative intensity difference between neighboring pixels. ### Claim 5 (depends on 4) The method of claim 4, wherein each frame score is based on a change in pitch of the audio portion from the audio portion of a preceding video frame. ### Claim 6 (depends on 1) The method of claim 1 including modifying at least one of the images of the key frames. ### Claim 7 A system for compressing a video clip comprising: a video analyzer having an output of frame scores for video frames of the video clip, each frame score based on at least one of an image attribute of an image of the video frame and an audio attribute of an audio portion of the video frame; a key frame selector having an output identifying key frames of the video frames having a frame score that exceeds a threshold frame score; and a video compiler configured to form a compressed video clip in which the images of non-key frames of the video clip are removed, the compressed video clip comprising the images of the key frames, the audio portions of the key frames and the audio portions of the non-key video frames. ### Claim 8 (depends on 7) The system of claim 7, wherein: each frame score is based on contrast measurement of the image of the video frame, which indicates a relative intensity difference between neighboring pixels of the image of the video frame; and the video analyzer assigns higher frame scores for the individual video frames whose images have a higher relative intensity difference between neighboring pixels than the video frames whose images have a lower relative intensity difference between neighboring pixels. ### Claim 9 (depends on 8) The system of claim 8, wherein each frame score is based on a change in pitch of the audio portion from the audio portion of a preceding video frame. ### Claim 10 A tangible computer storage medium including instructions executable by a microprocessor of a computing device for compressing a video clip, the instructions comprising: instructions for analyzing at least one of an image and an audio portion of individual video frames of the video clip; instructions for calculating frame scores for the individual video frames, each frame score based on at least one of an image attribute of the image of the video frame and an audio attribute of the audio portion of the video frame; instructions for identifying key frames having a frame score that exceeds a threshold frame score; and instructions for forming a compressed video clip comprising the images and audio portions of the key frames and the audio portions of the non-key frames, wherein the images of the non-key frames are not included in the compressed video clip. ### Claim 11 (depends on 10) The instructions of claim 10, wherein: each frame score is based on a contrast measurement of the image of the video frame, which indicates a relative intensity difference between neighboring pixels of the image of the video frame; and instructions for calculating frame scores for the individual video frames comprise instructions for assigning higher frame scores to the video frames whose images have a higher relative intensity difference between neighboring pixels than the video frames whose images have a lower relative intensity difference between neighboring pixels. ### Claim 12 (depends on 11) The instructions of claim 11, wherein each frame score is based on a change in pitch of the audio portion from the audio portion of a preceding video frame. ### Claim 13 (depends on 1) The method of claim 1, further comprising filling in image gaps of the non-key video frames of the compressed video clip with the images of the key frames. ### Claim 14 (depends on 1) The method of claim 1, wherein: each frame score is based on a histogram of the image of the video frame, from which a dynamic range for the image is obtained; and calculating frame scores for the individual video frames comprises assigning higher frame scores to the video frames whose images have a higher dynamic range than the video frames whose images have a lower dynamic range. ### Claim 15 (depends on 1) The method of claim 1, wherein: each frame score is based on a measurement of a magnitude of optical flow of the image of the video frame; and calculating frame scores for the individual video frames comprises assigning higher frame scores to the video frames whose images have a lower magnitude of optical flow than the video frames whose images have a higher magnitude of optical flow. ### Claim 16 (depends on 7) The system of claim 7, wherein the video compiler is further configured to fill in image gaps of the non-key video frames of the compressed video clip with the images of the key frames. ### Claim 17 (depends on 7) The system of claim 7, wherein the video compiler is further configured to extend a time code period for the images of the key frames into the non-key video frames of the compressed video clip. ### Claim 18 (depends on 7) The method of claim 7, wherein: each frame score is based on a histogram of the image of the video frame, from which a dynamic range for the image is obtained; and the video analyzer assigns higher frame scores for the individual video frames whose images have a higher dynamic range than the video frames whose images have a lower dynamic range. ### Claim 19 (depends on 7) The method of claim 7, wherein: each frame score is based on a measurement of a magnitude of optical flow of the image of the video frame; and the video analyzer assigns higher frame scores to the video frames whose images have a lower magnitude of optical flow than the video frames whose images have a higher magnitude of optical flow. ### Claim 20 (depends on 10) The instructions of claim 10, further comprising instructions for filling in image gaps of the non-key video frames of the compressed video clip with the images of the key frames. ### Claim 21 (depends on 10) The instructions of claim 10, wherein: each frame is based on a histogram of the image of the video frame, from which a dynamic range for the image is obtained; and instructions for calculating frame scores for the individual video frames comprise assigning higher frame scores to the video frames whose images have a higher dynamic range than the video frames whose images have a lower dynamic range. ### Claim 22 (depends on 10) The instructions of claim 10, wherein: each frame score is based on a measurement of a magnitude of optical flow of the image of the video frame; and instructions for calculating frame scores for the individual video frames comprise assigning higher frame scores to the video frames whose images have a lower magnitude of optical flow than the video frames whose images have a higher magnitude of optical flow.