Method and apparatus for recovering a three-dimensional scene from two-dimensional images

A method of recovering a three-dimensional scene from two-dimensional images, the method comprising:

• identifying a sequence of two-dimensional frames that include two-dimensional images;
• dividing the sequence of frames into segments, wherein a segment includes a plurality of frames and wherein dividing includes:
• identifying a base frame, identifying feature points in the base frame;
• and determining the segments such that every frame in a segment has at least a predetermined percentage of feature points identified in the base frame;
• for each segment, encoding the frames in the segment into at least two virtual frames that include a three-dimensional structure for the segment and an uncertainty associated with the segment and wherein encoding includes choosing at least two frames in the segment that are at least a threshold number of frames apart;
• for each of the at least two chosen frames, projecting a plurality of three-dimensional points into a corresponding virtual frame;
• and
• for each of the at least two chosen frames, projecting an uncertainty into the corresponding virtual frame.

A method of recovering a three-dimensional scene from a sequence of two-dimensional frames, comprising:

• (a) identifying at least a first base frame in a sequence of two-dimensional frames;
• (b) adding the at least first base frame to create a first segment of frames of the sequence;
• (c) selecting feature points in at least the first base frame in the first segment of frames in the sequence;
• (d) analyzing a next frame in the sequence to identify the selected feature points in the next frame;
• (e) determining a number of the selected feature points from the base frame that are also identified in the next frame;
• and
• (f) if the number of the selected feature points from the base frame that are also identified in the next frame is greater than or equal to a threshold number, adding the next frame to the first segment of frames of the sequence.

A computer-readable medium having computer-executable instructions for performing a method comprising:

• providing a sequence of two-dimensional frames;
• dividing the sequence into segments, each segment having at least two frames;
• calculating a partial model for each segment, wherein the partial model includes the same number of frames as the segment said partial model represents and wherein the partial model includes three-dimensional coordinates and camera pose, the camera pose comprising rotation and translation, for features within the frames;
• extracting virtual key frames from each partial model, the virtual key frames having three-dimensional coordinates for the frames and an uncertainty associated with the frames;
• and
• bundle adjusting the virtual key frames to obtain a complete three-dimensional reconstruction of the two-dimensional frames.

An apparatus for recovering a three-dimensional scene from a sequence of two-dimensional frames by segmenting the frames, comprising:

• means for capturing two-dimensional images;
• means for dividing the sequence into segments comprising at least two frames;
• means for calculating a partial model for each segment that includes three-dimensional coordinates and camera pose for features within the frames of the segment, the three-dimensional coordinates and camera pose being derived from the frames of the segment;
• means for extracting virtual key frames from each partial model;
• and
• means for bundle adjusting the virtual key frames to obtain a complete three-dimensional reconstruction of the two-dimensional frames.