Back to results
US 8,913,141 B2Granted
Edge-based electronic image stabilization
AssigneeGoPro, Inc.
InventorScott Patrick Campbell
Life of this patent
Estimating the term…
Abstract
An image stabilization system applies a “pinned-edge” or “soft pinned edge” image stabilization technique to digital video to compensate for unwanted camera motion in a captured video. In these stabilization techniques, a warping function is applied to an image frame to achieve a non-uniform shifting of depicted points in the image frame such that a reference point is stabilized with respect to a reference frame. In pinned-edge image stabilization, the final stabilized output video has the same dimensions as the pre-stabilized input video captured by the image sensor. In soft pinned-edge image stabilization, the pre-stabilized input video has slightly larger dimensions than the stabilized output video but these larger dimensions are still reduced compared to traditional electronic image stabilization.
Claims27
A method for stabilizing a digital video, the method comprising:
- receiving a reference video frame depicting a scene, the reference video frame depicting a reference point represented by one or more pixels at a reference pixel location;
- receiving a second frame depicting at least a portion of the scene, wherein movement of the reference point between the first and second frame results in the reference point being depicted at a different pixel location in the second frame;
- non-uniformly shifting non-edge points depicted in the second frame by an amount based on distances of the non-edge points from an edge of the second frame and applying zero shift to edge points depicted in the second frame;
- and
- generating a stabilized video comprising a sequence of frames including the reference frame and the stabilized second frame, wherein depicted motion of the reference point between the first frame and the stabilized second frame is reduced relative to depicted motion of the reference point between the first frame and the second frame.
2 more elements
A method for stabilizing a digital video, the method comprising:
- receiving the digital video comprising a sequence of frames depicting a scene, the sequence of frames including a reference frame and a second frame each depicting a common plurality of depicted points in the scene;
- determining a reference point from the common plurality of points, wherein the reference point corresponds to different pixel locations in the reference frame and the second frame;
- and
- applying a warping function to the second frame of the digital video to generate a warped second frame, wherein depicted points of the depicted scene in the warped second frame are shifted non-uniformly relative to corresponding depicted points in the second frame, and wherein applying the warping function decreases depicted movement of the reference point between the reference frame and the second frame, wherein applying the warping function to generate the warped second frame comprises:
- applying a first shift to a non-edge point, the first shift having a component in a first direction;
- and
- applying a second shift to an edge point on an edge of the second frame, wherein the non-edge point is shifted by a greater amount in the first direction than the edge point.
4 more elements
A method for stabilizing a digital video, the method comprising:
- receiving the digital video comprising a sequence of frames depicting a scene, the sequence of frames including a reference frame and a second frame each depicting a common plurality of depicted points in the scene;
- determining a reference point from the common plurality of points, wherein the reference point corresponds to different pixel locations in the reference frame and the second frame;
- and
- applying a warping function to the second frame of the digital video to generate a warped second frame, wherein depicted points of the depicted scene in the warped second frame are shifted non-uniformly relative to corresponding depicted points in the second frame, and wherein applying the warping function decreases depicted movement of the reference point between the reference frame and the second frame, wherein applying the warping function comprises:
- applying a first shift to a non-edge point, the first shift having a component in a first direction;
- and
- applying no shift in the first direction to an edge point on an edge of the second frame.
4 more elements
A non-transitory computer-readable storage medium storing computer-executable instructions for stabilizing a digital video, the computer-executable instructions when executed by one more processors cause the one or more processors to performs steps including:
- receiving the digital video comprising a sequence of frames depicting a scene, the sequence of frames including a reference frame and a second frame each depicting a common plurality of depicted points in the scene;
- determining a reference point from the common plurality of points, wherein the reference point corresponds to different pixel locations in the reference frame and the second frame;
- and
- applying a warping function to the second frame of the digital video to generate a warped second frame, wherein depicted points of the depicted scene in the warped second frame are shifted non-uniformly relative to corresponding depicted points in the second frame, and wherein applying the warping function decreases depicted movement of the reference point between the reference frame and the second frame, wherein applying the warping function to generate the warped second frame comprises:
- applying a first shift to a non-edge point, the first shift having a component in a first direction;
- and
- applying a second shift to an edge point on an edge of the second frame, wherein the non-edge point is shifted by a greater amount in the first direction than the edge point.
4 more elements
A non-transitory computer-readable storage medium storing computer-executable instructions for stabilizing a digital video, the computer-executable instructions when executed by one more processors cause the one or more processors to performs steps including:
- receiving the digital video comprising a sequence of frames depicting a scene, the sequence of frames including a reference frame and a second frame each depicting a common plurality of depicted points in the scene;
- determining a reference point from the common plurality of points, wherein the reference point corresponds to different pixel locations in the reference frame and the second frame;
- and
- applying a warping function to the second frame of the digital video to generate a warped second frame, wherein depicted points of the depicted scene in the warped second frame are shifted non-uniformly relative to corresponding depicted points in the second frame, and wherein applying the warping function decreases depicted movement of the reference point between the reference frame and the second frame, wherein applying the warping function comprises:
- applying a first shift to a non-edge point, the first shift having a component in a first direction;
- and
- applying no shift in the first direction to an edge point on an edge of the second frame.
4 more elements
An image stabilization system for capturing and stabilizing digital video, the image stabilization system comprising:
- a camera system for capturing digital video;
- one or more processors;
- and
- a computer-readable storage medium storing computer-executable instructions for stabilizing the digital video, the computer-executable instructions when executed by the one more processors cause the one or more processors to performs steps including:
- receiving the digital video comprising a sequence of frames depicting a scene, the sequence of frames including a reference frame and a second frame each depicting a common plurality of depicted points in the scene;
- determining a reference point from the common plurality of points, wherein the reference point corresponds to different pixel locations in the reference frame and the second frame;
- and
- applying a warping function to the second frame of the digital video to generate a warped second frame, wherein depicted points of the depicted scene in the warped second frame are shifted non-uniformly relative to corresponding depicted points in the second frame, and wherein applying the warping function decreases depicted movement of the reference point between the reference frame and the second frame, wherein applying the warping function to generate the warped second frame comprises:
- applying a first shift to a non-edge point, the first shift having a component in a first direction;
- and
- applying a second shift to an edge point on an edge of the second frame, wherein the non-edge point is shifted by a greater amount in the first direction than the edge point.
8 more elements
An image stabilization system for capturing and stabilizing digital video, the image stabilization system comprising:
- a camera system for capturing digital video;
- one or more processors;
- and
- a computer-readable storage medium storing computer-executable instructions for stabilizing the digital video, the computer-executable instructions when executed by the one more processors cause the one or more processors to performs steps including:
- receiving the digital video comprising a sequence of frames depicting a scene, the sequence of frames including a reference frame and a second frame each depicting a common plurality of depicted points in the scene;
- determining a reference point from the common plurality of points, wherein the reference point corresponds to different pixel locations in the reference frame and the second frame;
- and
- applying a warping function to the second frame of the digital video to generate a warped second frame, wherein depicted points of the depicted scene in the warped second frame are shifted non-uniformly relative to corresponding depicted points in the second frame, and wherein applying the warping function decreases depicted movement of the reference point between the reference frame and the second frame, wherein applying the warping function comprises:
- applying a first shift to a non-edge point, the first shift having a component in a first direction;
- and
- applying no shift in the first direction to an edge point on an edge of the second frame.
8 more elements
Dependencies are inferred from the claim text.