Methods and systems for detecting and combining structural features in 3D reconstruction

A system for updating camera poses, the system comprising:

• a depth camera;
• and
• one or more processors communicatively coupled to the depth camera, wherein the one or more processors are configured to perform operations including:
• obtaining, by the depth camera, a set of captured depth maps, each of the captured depth maps being associated with a physical camera pose, the set of captured depth maps comprising a scene;
• detecting a first shape and a second shape present in the scene, the first shape characterized by a first dimension and/or first location/orientation, and the second shape characterized by a second dimension and/or second location/orientation;
• creating a first 3D mesh for the first shape;
• creating a first virtual camera associated with the first 3D mesh in a first local reference frame;
• rendering a first depth map, the first depth map associated with the first virtual camera;
• creating a second 3D mesh for the second shape;
• creating a second virtual camera associated with the second 3D mesh in a second local reference frame;
• rendering a second depth map, the second depth map associated with the second virtual camera;
• identifying a subset of captured depth maps, each of the captured depth maps in the subset comprising at least a first portion of the first shape or a second portion of the second shape;
• and
• updating the physical camera poses associated with the subset of captured depth maps to provide updated physical camera poses by jointly solving, using the first shape and the second shape, for the physical camera poses, the first dimension and first location/orientation, and the second dimension and second location/orientation by optimizing an alignment between the first depth map, the second depth map, and a subset of the set of captured depth maps.

A system for forming a reconstructed 3D mesh, the system comprising:

• a depth camera;
• and
• one or more processors communicatively coupled to the depth camera, wherein the one or more processors are configured to perform operations comprising:
• obtaining, by the depth camera, a set of captured depth maps associated with a scene;
• performing an initial camera pose alignment associated with each captured depth map of the set of captured depth maps;
• overlaying the set of captured depth maps in a reference frame;
• detecting one or more shapes in the overlaid set of captured depth maps, thereby providing one or more detected shapes;
• updating, using the one or more detected shapes, the initial camera pose alignment based on an overlap between the overlaid set of captured depth maps and the one or more detected shapes to provide a shape-aware camera pose alignment associated with each captured depth map of the set of captured depth maps;
• performing, using the one or more detected shapes, shape-aware volumetric fusion;
• and
• forming the reconstructed 3D mesh associated with the scene.