Online calibration of 3D scan data from multiple viewpoints

A method, comprising:

• receiving a set of depth scans and a corresponding set of color images of a scene comprising a human-object as part of a foreground of the scene;
• extracting a first three-dimensional (3D) representation of the foreground based on a first depth scan of the set of depth scans, wherein the first 3D representation is associated with a first viewpoint in a 3D environment;
• spatially aligning the extracted first 3D representation with a second 3D representation of the foreground, wherein the second 3D representation is associated with a second viewpoint in the 3D environment;
• updating the spatially aligned first 3D representation based on the corresponding set of color images and a set of structural features of the human-object, as a human-prior;
• extracting a third 3D representation of a ground surface in the scene based on the first depth scan associated with the first viewpoint;
• spatially aligning the extracted third 3D representation of the ground surface with a fourth 3D representation of the ground surface, the fourth 3D representation being associated with the second viewpoint of the scene;
• updating the spatially aligned first 3D representation further based on the spatial alignment of the extracted third 3D representation with the fourth 3D representation;
• and
• reconstructing a 3D mesh of the human-object based on the updated first 3D representation of the foreground.

A non-transitory computer-readable medium having stored thereon, computer-executable instructions that when executed by a calibration system, causes the calibration system to execute operations, the operations comprising:

• receiving a set of depth scans and a corresponding set of color images of a scene comprising a human-object as part of a foreground of the scene;
• extracting a first three-dimensional (3D) representation of the foreground based on a first depth scan of the set of depth scans, wherein the first 3D representation is associated with a first viewpoint in a 3D environment;
• spatially aligning the extracted first 3D representation with a second 3D representation of the foreground, wherein the second 3D representation is associated with a second viewpoint in the 3D environment;
• updating the spatially aligned first 3D representation based on the corresponding set of color images and a set of structural features of the human-object, as a human-prior;
• extracting a third 3D representation of a ground surface in the scene based on the first depth scan associated with the first viewpoint;
• spatially aligning the extracted third 3D representation of the ground surface with a fourth 3D representation of the ground surface, the fourth 3D representation being associated with the second viewpoint of the scene;
• updating the spatially aligned first 3D representation further based on the spatial alignment of the extracted third 3D representation with the fourth 3D representation;
• and
• reconstructing a 3D mesh of the human-object based on the updated first 3D representation of the foreground.