US 2015/0317802 A1 · Opt-keyframe Reconstruction for Robust Video-based Structure from Motion

Abstract

A non-keyframe reconstruction technique is described for selecting and reconstructing keyframes that have not yet been included in a reconstruction of an input image sequence to provide a better reconstruction in a structure from motion (SFM) technique. The technique may, for example, be used in an adaptive reconstruction algorithm implemented by a general SFM technique. This technique may add and reconstruct non-keyframes to a set of keyframes already generated by an initialization technique and reconstructed by adaptive and optimization techniques for iteratively selecting and reconstructing additional keyframes. Camera motion and intrinsic parameters may be computed for non-keyframes by optimizing a cost function. Output of the non-keyframe reconstruction technique may include at least camera intrinsic parameters and Euclidean motion parameters for the images in the input image sequence.

Claims20

Dependent claims

A method, comprising:

performing, by one or more computing devices:
obtaining an input image sequence including a plurality of frames;
obtaining a reconstruction of camera motion and camera intrinsic parameters for the image sequence, the reconstruction including a set of keyframes selected from the plurality of frames of the image sequence;
selecting one of the plurality of frames of the image sequence that is not included in the reconstruction;
reconstructing camera motion and camera intrinsic parameters for the selected frame according to the reconstruction and at least a portion of a set of point trajectories for the image sequence, the reconstructing adding the selected frame to the reconstruction;
repeating the selecting and the reconstructing until each of the plurality of frames of the image sequence is included in the reconstruction;
and
performing a global optimization of each of the frames in the reconstruction.

5 more elements

A system, comprising:

one or more processors;
and
a memory comprising program instructions, the program instructions executable by at least one of the one or more processors to:
obtain an input image sequence including a plurality of frames;
obtain a reconstruction of camera motion and camera intrinsic parameters for the image sequence, the reconstruction including a set of keyframes selected from the plurality of frames of the image sequence;
select one of the plurality of frames of the image sequence that is not included in the reconstruction;
reconstruct camera motion and camera intrinsic parameters for the selected frame according to the reconstruction and at least a portion of a set of point trajectories for the image sequence, the reconstructing adding the selected frame to the reconstruction;
repeat the selecting and the reconstructing until each of the plurality of frames of the image sequence is included in the reconstruction;
remove frames from the reconstruction that do not appear in a largest contiguous subset of frames of the reconstruction according to the set of point trajectories for the image sequence;
and
output at least the camera intrinsic parameters and Euclidean motion parameters for the images in the input image sequence.

8 more elements

A non-transitory computer-readable storage medium storing program instructions, the program instructions being computer-executable to implement:

obtaining an input image sequence including a plurality of frames;
obtaining a reconstruction of camera motion and camera intrinsic parameters for the image sequence, the reconstruction including a set of keyframes selected from the plurality of frames of the image sequence;
selecting one of the plurality of frames of the image sequence that is not included in the reconstruction;
reconstructing camera motion and camera intrinsic parameters for the selected frame according to the reconstruction and at least a portion of a set of point trajectories for the image sequence, the reconstructing adding the selected frame to the reconstruction;
repeating the selecting and the reconstructing until each of the plurality of frames of the image sequence is included in the reconstruction;
and
outputting at least the camera intrinsic parameters and Euclidean motion parameters for the images in the input image sequence.

4 more elements

Dependencies are inferred from the claim text.

Opt-keyframe Reconstruction for Robust Video-based Structure from Motion

Life of this patent

Opt-keyframe Reconstruction for Robust Video-based Structure from Motion

Life of this patent

Abstract

Claims20