Back to results
US 8,711,233 B2Granted
Video stabilization
AssigneeSkype
InventorsAndrei Jefremov, David Zhao, Pontus Carlsson
Life of this patent
Estimating the term…
Abstract
Method, device and computer program product for stabilizing a video signal. A plurality of frames of the video signal are captured using a camera. A motion sensor associated with the camera is used to generate a plurality of samples representing motion of the camera. The samples are used to determine a displacement of the camera between a first time and a second time, wherein the first time corresponds to an exposure time midpoint of a first frame of the video signal and the second time corresponds to an exposure time midpoint of a second frame of the video signal. The determined displacement is used to compensate for motion in the video signal between the first and second frames caused by the motion of the camera, to thereby stabilize the video signal.
Claims21
A method of stabilizing a video signal, the method comprising:
- capturing a plurality of frames of the video signal using a camera;
- using a motion sensor associated with the camera to generate a plurality of samples representing motion of the camera;
- using the samples to determine a displacement of the camera between a first time and a second time, wherein the first time corresponds to an exposure time midpoint of a first frame of the video signal and the second time corresponds to an exposure time midpoint of a second frame of the video signal;
- and
- using the determined displacement to compensate for motion in the video signal between the first and second frames caused by the motion of the camera, thereby stabilizing the video signal by:
- determining a pixel displacement representing motion in the video signal between the first and second frames caused by the determined displacement of the camera;
- filtering the pixel displacement by determining an accumulated pixel displacement based on said determined pixel displacement for the second frame and determining a filtered accumulated pixel displacement for the second frame based on a weighted sum of the determined accumulated pixel displacement for the second frame and a filtered accumulated pixel displacement for the first frame;
- and
- shifting the image of at least one of the first and second frames in accordance with the filtered pixel displacement to thereby stabilize the video signal.
6 more elements
A device configured to stabilize a video signal, the device comprising:
- a camera configured to capture a plurality of frames of the video signal;
- a motion sensor, associated with the camera, configured to generate a plurality of samples representing motion of the camera, wherein the motion sensor is a rotational motion sensor and the motion of the camera is rotational motion;
- a displacement determining block configured to use the samples to determine a displacement of the camera between a first time and a second time, wherein the displacement of the camera is an angular displacement of the camera, and wherein the first time corresponds to an exposure time midpoint of a first frame of the video signal and the second time corresponds to an exposure time midpoint of a second frame of the video signal;
- and
- a motion compensation block configured to use the determined displacement to compensate for motion in the video signal between the first and second frames caused by the motion of the camera, to thereby stabilize the video signal, the motion compensation block further configured to:
- determine an angular velocity of the camera using the samples;
- and
- integrate the determined angular velocity over time from the first time to the second time to thereby determine the angular displacement of the camera between the first time and the second time, wherein to integrate the determined angular velocity includes interpolating the angular velocity between the times at which the samples are generated;
- the device further configured to delay the video signal thereby allowing the angular velocity of the camera at the second time to be determined by said interpolating the angular velocity between the times at which the samples are generated.
6 more elements
A computing device comprising:
- one or more processors;
- one or more computer-readable storage memories embodying computer-readable instructions which, when executed by the one or more processors, cause the computing device to perform operations for stabilizing a video signal comprising:
- capturing a plurality of frames of the video signal using a camera;
- using a motion sensor associated with the camera to generate a plurality of samples representing motion of the camera, wherein the motion sensor is a rotational motion sensor and wherein the motion of the camera is rotational motion;
- using the samples to determine a displacement of the camera between a first time and a second time, wherein the displacement of the camera is an angular displacement of the camera, and wherein the first time corresponds to an exposure time midpoint of a first frame of the video signal and the second time corresponds to an exposure time midpoint of a second frame of the video signal, the determining comprising:
- determining an angular velocity of the camera using the samples;
- and
- integrating the determined angular velocity over time from the first time to the second time to thereby determine the angular displacement of the camera between the first time and the second time, wherein the angular displacement of the camera between the first time and the second time is given by the equation
- Δθ
- =
- ∫
- t
- 1
- -
- 0.5
-
- e
- 1
- t
- 2
- -
- 0.5
-
- e
- 2
-
- ω
-
- (
- t
- )
-
- ⅆ
- t
- where Δθ is the angular displacement of the camera, t1 is end time of the first frame, t2 is end time of the second frame, e1 is the exposure time of the first frame, e2 is the exposure time of the second frame and ω(t) is the angular velocity of the camera determined using the samples;
- and
- using the determined displacement to compensate for motion in the video signal between the first and second frames caused by the motion of the camera, to thereby stabilize the video signal.
34 more elements
Dependencies are inferred from the claim text.