Interactive construction of 3D models from panoramic images employing hard and soft constraint characterization and decomposing techniques

A computer-implemented process to construct a model of a 3D scene from a single panoramic view of the scene, comprising using a computer to perform the following steps:

• displaying an image of the single panoramic view of the 3D scene as a screen image;
• recovering a camera orientation matrix of the single panoramic view based on lines associated with at least one plane of the 3D scene as specified by a user of the computer-implemented process;
• characterizing the 3D scene as at least one set of connected planes, each set being subject to a series of geometric constraints;
• estimating plane normals and line directions for planes in the 3D scene which are to be modeled using the camera orientation matrix exclusive of those provided by the user;
• recovering a camera translation using the camera orientation matrix, wherein the camera translation is the difference between the camera projection center and the coordinate origin of the 3D scene;
• estimating unknown plane distances and vertex point locations, for each plane in the 3D scene which is to be modeled, by solving a system of linear geometric constraint equations created using the estimated plane normals and line directions, as well as user designated parameters comprising points, plane normals, line directions, plane distances and parallel planes;
• constructing a model of the 3D scene consisting of at least one set of connected planes based on the plane normal and plane distance, and/or the vertex point locations, of each plane in the set.

A system for constructing a model of a 3D scene from a single panoramic view of the scene, comprising:

• a general purpose computing device;
• and
• a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,
• display an image of the single panoramic view of the 3D scene on a computer monitor screen,
• recover a camera orientation matrix of the single panoramic view based on lines associated with at least one plane of the 3D scene as specified by a user of the computer-implemented process,
• characterize the 3D scene as at least one set of connected planes, each set being subject to a series of geometric constraints,
• estimate plane normals and line directions for planes in the 3D scene which are to be modeled using the camera orientation matrix,
• recover a camera translation using the camera orientation matrix, wherein the camera translation is the difference between the camera projection center and the coordinate origin of the 3D scene,
• estimate unknown plane distances and vertex point locations, for each plane in the 3D scene which is to be modeled, by solving a system of linear geometric constraint equations created using the estimated plane normals and line directions, as well as user designated parameters comprising points, plane normals, line directions, plane distances and parallel planes, wherein the constraint equations have been characterized as being a hard constraint whenever the equation includes at least one of the user designated parameters and a soft constraint whenever the equation does not include a user designated parameter, and
• constructing a model of the 3D scene consisting of at least one set of connected planes based on the plane normal and plane distance, and/or the vertex point locations, of each plane in the set.

A system for constructing a model of a 3D scene from a single panoramic view of the scene, comprising:

• a general purpose computing device;
• and
• a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,
• display an image of the single panoramic view of the 3D scene on a computer monitor screen,
• recover a camera orientation matrix of the single panoramic view based on lines associated with at least one plane of the 3D scene as specified by a user of the computer-implemented process,
• characterize the 3D scene as at least one set of connected planes, each set being subject to a series of geometric constraints,
• estimate plane normals and line directions for planes in the 3D scene which are to be modeled using the camera orientation matrix,
• recover a camera translation using the camera orientation matrix, wherein the camera translation is the difference between the camera projection center and the coordinate origin of the 3D scene,
• estimate unknown plane distances and vertex point locations, for each plane in the 3D scene which is to be modeled, by solving a system of linear geometric constraint equations created using the estimated plane normals and line directions, as well as user designated parameters comprising points, plane normals, line directions, plane distances and parallel planes,
• construct a model of the 3D scene consisting of at least one set of connected planes based on the plane normal and plane distance, and/or the vertex point locations, of each plane in the set.

A computer-readable memory for constructing a model of a 3D scene from a single panoramic view of the scene, comprising:

• a computer-readable storage medium;
• and
• a computer program comprising program modules stored in the storage medium, wherein the storage medium is so configured by the computer program that it causes the computer to,
• display an image of the single panoramic view of the 3D scene an a computer monitor screen,
• recover a camera orientation matrix of the single panoramic view based on lines associated with at least one plane of the 3D scene as specified by a user of the computer-implemented process,
• characterize the 3D scene as at least one set of connected planes, each set being subject to a series of geometric constraints,
• estimate plane normals and line directions for planes in the 3D scene which are to be modeled using the camera orientation matrix,
• recover a camera translation using the camera orientation matrix, wherein the camera translation is the difference between the camera projection center and the coordinate origin of the 3D scene,
• estimate unknown plane distances and vertex point locations, for each plane in the 3D scene which is to be modeled, by solving a system of linear geometric constraint equations created using the estimated plane normals and line directions, as well as user designated parameters comprising points, plane normals, line directions, plane distances and parallel planes, wherein the constraint equations have been characterized as being a hard constraint whenever the equation includes at least one of the user designated parameters and a soft constraint whenever the equation does not include a user designated parameter, and
• constructing a model of the 3D scene consisting of at least one set of connected planes based on the plane normal and plane distance, and/or the vertex point locations, of each plane in the set.

A computer-readable memory for constructing a model of a 3D scene from a single panoramic view of the scene, comprising:

• a computer-readable storage medium;
• and
• a computer program comprising program modules stored in the storage medium, wherein the storage medium is so configured by the computer program that it causes the computer to,
• display an image of the single panoramic view of the 3D scene on a computer monitor screen,
• recover a camera orientation matrix of the single panoramic view based on lines associated with at least one plane of the 3D scene as specified by a user of the computer implemented process,
• characterize the 3D scene as at least one set of connected planes, each set being subject to a series of geometric constraints,
• estimate plane normals and line directions for planes in the 3D scene which are to be modeled using the camera orientation matrix,
• recover a camera translation using the camera orientation matrix, wherein the camera translation is the difference between the camera projection center and the coordinate origin of the 3D scene,
• estimate unknown plane distances and vertex point locations, for each plane in the 3D scene which is to be modeled, by solving a system of linear geometric constraint equations created using the estimated plane normals and line directions, as well as user designated parameters comprising points, plane normals, line directions, plane distances and parallel planes,
• construct a model of the 3D scene consisting of at least one set of connected planes based on the plane normal and plane distance, and/or the vertex point locations, of each plane in the set et.

A system for constructing a model of a 3D scene from a single panoramic view of the scene, comprising:

• a general purpose computing device;
• an interface for dynamically interfacing a user to the computing device, wherein the user is provided with a display of an image of the single panoramic view of the 3D scene on a computer monitor screen, the user then specifying features on the screen image;
• a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,
• recover a camera orientation matrix of the single panoramic view based on lines associated with at least one plane of the 3D scene as specified by a user of the computer-implemented process,
• characterize the 3D scene as at least one set of connected planes, each set being subject to a series of geometric constraints,
• estimate plane normals and line directions for planes in the 3D scene which are to be modeled using the camera orientation matrix,
• recover a camera translation using the camera orientation matrix, wherein the camera translation is the difference between the camera projection center and the coordinate origin of the 3D scene,
• estimate unknown plane distances and vertex point locations, for each plane in the 3D scene which is to be modeled, by solving a system of linear geometric constraint equations created using the estimated plane normals and line directions, as well as user designated parameters comprising points, plane normals, line directions, plane distances and parallel planes, wherein the constraint equations have been characterized as being a hard constraint whenever the equation includes at least one of the user designated parameters and a soft constraint whenever the equation does not include a user designated parameter, and
• constructing a model of the 3D scene consisting of at least one set of connected planes based on the plane normal and plane distance, and/or the vertex point locations, of each plane in the set.

A system for constructing a model of a 3D scene from a single panoramic view of the scene, comprising:

• a general purpose computing device;
• an interface for dynamically interfacing a user to the computing device , wherein the user is provided with a display of an image of the single panoramic view of the 3D scene on a computer monitor screen, the user then specifying features on the screen image;
• a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,
• recover a camera orientation matrix of the single panoramic view based on lines associated with at least one plane of the 3D scene as specified by a user of the computer-implemented process,
• characterize the 3D scene as at least one set of connected planes, each set being subject to a series of geometric constraints,
• estimate plane normals and line directions for planes in the 3D scene which are to be modeled using the camera orientation matrix,
• recover a camera translation using the camera orientation matrix, wherein the camera translation is the difference between the camera projection center and the coordinate origin of the 3D scene,
• estimate unknown plane distances and vertex point locations, for each plane in the 3D scene which is to be modeled, by solving a system of linear geometric constraint equations created using the estimated plane normals and line directions, as well as user designated parameters comprising points, plane normals, line directions, plane distances and parallel planes,
• construct a model of the 3D scene consisting of at least one set of connected planes based on the plane normal and plane distance, and/or the vertex point locations, of each plane in the set.

A computer-implemented process to construct a model of a 3D scene from a single panoramic image of the scene, comprising using a computer to perform the following process action:

• displaying the single panoramic image of the scene to a user;
• recovering a camera orientation matrix of the single panoramic image based on lines associated with at least one plane of the 3D scene as specified by a user of the computer-implemented process using the displayed image;
• characterizing the 3D scene as at least one set of connected planes, each set being subject to a series of geometric constraints;
• estimating plane normals and line directions for planes in the 3D scene which are to be modeled using the camera orientation matrix,
• recovering a camera translation using the camera orientation matrix, wherein the camera translation is the difference between the camera projection center and the coordinate origin of the 3D scene;
• estimating unknown plane distances and vertex point locations, for each plane in the 3D scene which is to be modeled, by solving a system of linear geometric constraint equations created using the estimated plane normals and line directions, as well as user designated parameters, wherein said parameters comprise at least one point plane normal, line direction, plane distance or pair of parallel planes specified by the user using said displayed image of the scene;
• constructing a model of the 3D scene consisting of at least one set of connected planes based on the plane normal and plane distance, and/or the vertex point locations, of each plane in the set.