Interior photographic documentation of architectural and industrial environments using 360 panoramic videos

A computer system comprising a processor coupled to a storage device, the storage device storing processor-implemented instructions for producing a photographic record of a site, said instructions for causing said processor to perform steps including:

• extract a plurality of 360 images from a 360 panoramic video sequence, determine three-dimensional (“3D”) poses of the extracted plurality of 360 images relative to each other and to a plan of the site, and produce a model of the site from the extracted plurality of 360 images, said model aligned to a plan of the site, wherein:
• the 3D poses of the extracted plurality of 360 images are determined by first determining an approximate pose based on features sequentially tracked between sequential images of the 360 panoramic video sequence and then refined based on features matched across a subset of images selected from the 360 panoramic video sequence;
• and
• if the plan of the site is a two-dimensional (“2D”) drawing, alignment to the plan of the site is obtained by determining correspondences between lines in the 2D drawing and lines derived from a sparse 3D point cloud produced from the subset of images selected from the 360 panoramic video sequence, said derivation performed by projecting 3D planes from the 3D point cloud determined to be perpendicular to a ground plane into lines on the ground plane, otherwise, if the plan of the site is a 3D computer aided design (“CAD”) or building information model (“BIM”), alignment to the plan of the site is obtained by determining correspondences between points or planes derived from the sparse 3D point cloud and planes derived from the 3D CAD or BIM using ground plane projections of said points or planes derived from the 3D point cloud and ground plane projections of said planes derived from the 3D CAD or BIM.

A system, comprising a camera for producing a 360 video and a computer system coupled to receive said 360 video from said camera, said computer system including a processor and a storage device, the storage device storing processor-implemented instructions for producing a photographic record of a site from said 360 video, said instructions for causing said processor to perform steps including:

• extract a plurality of 360 images from the video, determine three-dimensional (“3D”) poses of the extracted plurality of 360 images relative to each other and to a plan of the site, and produce a model of the site from the extracted plurality of 360 images, said model aligned to a plan of the site, wherein:
• the 3D poses of the extracted plurality of 360 images are determined by first determining an approximate pose based on features sequentially tracked between sequential images of the 360 panoramic video sequence and then refined based on features matched across a subset of images selected from the 360 panoramic video sequence;
• and
• if the plan of the site is a two-dimensional (“2D”) drawing, alignment to the plan of the site is obtained by determining correspondences between lines in the 2D drawing and lines derived from a sparse 3D point cloud produced from the subset of images selected from the 360 panoramic video sequence, said derivation performed by projecting 3D planes from the 3D point cloud determined to be perpendicular to a ground plane into lines on the ground plane, otherwise, if the plan of the site is a 3D computer aided design (“CAD”) or building information model (“BIM”), alignment to the plan of the site is obtained by determining correspondences between points or planes derived from the sparse 3D point cloud and planes derived from the 3D CAD or BIM using ground plane projections of said points or planes derived from the 3D point cloud and ground plane projections of said planes derived from the 3D CAD or BIM.