US 8,350,850 B2Granted

Using photo collections for three dimensional modeling

AssigneeMicrosoft Corporation

InventorsDrew Steedly, Richard Szeliski, Sudipta Sinha, Maneesh Agrawala

Life of this patent

Estimating the term…

Loading patent…

Abstract

A collection of photos and a three-dimensional reconstruction of the photos are used to construct and texture a mesh model. In one embodiment, a first digital image of a first view of a real world scene is analyzed to identify lines in the first view. Among the lines, parallel lines are identified. A three-dimensional vanishing direction in a three-dimensional space is determined based on the parallel lines and an orientation of the digital image in the three-dimensional space. A plane is automatically generated by fitting the plane to the vanishing direction. A rendering of a three-dimensional model with the plane is displayed. Three-dimensional points corresponding to features common to the photos may be used to constrain the plane. The photos may be projected onto the model to provide visual feedback when editing the plane. Furthermore, the photos may be used to texture the model.

Claims13

Dependent claims

A computer implemented method of fitting a plane for a three-dimensional model, the model comprising vertices interconnected by line segments, the plane and model being represented by data in computer memory, the method comprising:

accessing:
a collection of digital photos and a cloud of points, where the photos are of a real world scene, where the photos are three-dimensionally arranged in a three-dimensional space containing the three-dimensional model, where the points are also arranged in the three-dimensional space, and where each point in the cloud corresponds to a feature of the scene that exists in at least two of the photos;
for each digital photo, for a given digital photo, computing and storing a vanishing direction specific to the given digital photo by analyzing the given digital photo to identify a corresponding vanishing point specific to the given digital photo, and using the vanishing point and the orientation of the given digital photo in the three-dimensional space to compute the vanishing direction specific to the given digital photo;
receiving user input and according to the user input selecting, from among the first and second vanishing directions, the first vanishing direction;
arranging the plane in the three-dimensional space by fitting the plane to at least the vanishing direction specific to the first digital photo and to at least one of the points in the plurality of points;
and
displaying a rendering of the model according to the fitted plane.

4 more elements

One or more volatile and/or non-volatile computer readable media storing information to enable a computer to perform a process, wherein the computer readable media is not a signal, the process comprising:

analyzing a first digital image of a first view of a real world scene to identify lines in the first view of the scene identifying thereamong a first set of lines that are determined to be parallel in the first view of the scene;
analyzing a second digital image of a second view of a real world scene to identify lines in the second view of the scene and identifying thereamong a second set of lines that are determined to be parallel in the second view of the scene;
determining a first three-dimensional vanishing direction in a three-dimensional space based on the first set of parallel lines and an orientation of the first digital image in the three-dimensional space, the determining using parallel lines only from the first digital image;
determining a second three-dimensional vanishing direction in the three-dimensional space based on the second set of parallel lines and an orientation of the second digital image in the three-dimensional space, the determining using parallel lines only from the second digital image;
automatically fitting a plane of a model in the three-dimensional space by determining which of the vanishing directions provides a more optimal fit of the plane to the three-dimensional model and then constraining the plane to both the determined one of vanishing directions and to a point in the three-dimensional space;
and
displaying a rendering of a three-dimensional model in the three-dimensional space, where the model includes the fitted plane.

4 more elements

Dependencies are inferred from the claim text.