Holocam systems and methods

A system comprising:

• an acquisition module that receives visible image data and range data of a scene from two or more vantage points via one or more range data acquisition devices and one or more visible image acquisition devices;
• a model creation module that produces, for a vantage point, a three-dimensional (3D) model by combining range data and visible image data from that vantage point;
• and
• a scene registration module, communicatively coupled to the model creation module, that performs scene registration to acquire geometric transformations for merging two or more 3D models of the scene from different vantage points into a source-independent canonical space;
• wherein the scene registration module is configured to perform the steps comprising:
• performing point cloud generation from data about the scene taken from a vantage point;
• detecting planar surfaces in the point cloud;
• identifying a floor plane from among the detected planar surfaces;
• and
• estimating a canonical transformation that transforms the point cloud so that points corresponding to the floor plane are mapped to plane parallel to a z=0 plane and vertical direction is correct.

A computer-implemented method comprising:

• receiving color and depth of a scene from a plurality of vantage points from one or more input devices;
• producing, for each vantage point, a three-dimensional (3D) model of data by combining depth data information and visible image data;
• defining a source-independent canonical space which is used to register two or more 3D models into that space;
• and
• performing scene registration to acquire geometrical transformations to merge two or more 3D models of the same scene from their corresponding different vantage points into a combined 3D model;
• wherein the scene registration includes steps of:
• performing point cloud generation from data about the scene taken from a vantage point;
• detecting planar surfaces in the point cloud;
• identifying a floor plane from among the detected planar surfaces;
• and
• estimating a canonical transformation that transforms the point cloud so that points corresponding to the floor plane are mapped a plane parallel to a z=0 plane and vertical direction is correct.

A computer-implemented method comprising:

• receiving color and depth of a scene from a plurality of vantage points from one or more input devices;
• producing, for each vantage point, a three-dimensional (3D) model of data by combining depth data information and visible image data;
• defining a source-independent canonical space which is used to register two or more 3D models into that space;
• and
• performing scene registration to acquire geometrical transformations to merge two or more 3D models of the same scene from their corresponding different vantage points into a combined 3D model;
• wherein the scene registration is user-assisted and comprises the steps of:
• responsive to an input device having part of its sensors affected by one or more conditions in its field of view, generating an image mask for the input device to mask the affected part of its sensors;
• positioning a point acquisition device within the field of view of at least two input devices;
• using one or more signals from the point acquisition device to generate points which are used to find correspondences of points between data captured by at least two input devices;
• using the correspondences of points to identify three-dimensional points in each of a plurality of image data of a scene;
• and
• using at least some of the identified three-dimensional points to obtain a scene registration matrix.

A computer-implemented method comprising:

• receiving color and depth of a scene from a plurality of vantage points from one or more input devices;
• producing, for each vantage point, a three-dimensional (3D) model of data by combining depth data information and visible image data;
• defining a source-independent canonical space which is used to register two or more 3D models into that space;
• and
• performing scene registration to acquire geometrical transformations to merge two or more 3D models of the same scene from their corresponding different vantage points into a combined 3D model;
• wherein scene registration further includes performing scene reregistration to improve one or more geometric transformations;
• wherein scene reregistration comprises the step of:
• [a] selecting as a basis a point cloud generated from data of the scene taken from a vantage point;
• [b] reprojecting point clouds generated from data of the scene taken from different vantage points onto the basis and treating the aggregation of the resulting combined points clouds as one virtual point cloud;
• [c] solving for rigid body movement and producing rotation (R*) and translation (T*) values;
• [d] using the produced rotation (R*) and translation (T*) values to update scene registration information;
• [e] responsive to a stop condition being reached, selecting a best set of scene registration information based on one or more evaluation metrics;
• and
• [f] responsive to a stop condition not being reached, iterating by returning to step [a].

A system comprising

• one or more processors;
• and
• a non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by the one or more processors, causes steps to be performed comprising:
• receiving color and depth of a scene from a plurality of vantage points from one or more input devices;
• producing, for each vantage point, a three-dimensional (3D) model of data by combining depth data information and visible image data;
• defining a source-independent canonical space which is used to register two or more 3D models into that space;
• and
• performing scene registration to acquire geometrical transformations to merge two or more 3D models of the same scene from their corresponding different vantage points into a combined 3D model;
• wherein scene registration comprises the steps of:
• performing point cloud generation from data about the scene taken from a vantage point;
• detecting planar surfaces in the point cloud;
• identifying a floor plane from among the detected planar surfaces;
• estimating a canonical transformation that transforms the point cloud so that points corresponding to the floor plane are mapped to a plane parallel to a z=0 plane and vertical direction is correct.

A system comprising

• one or more processors;
• and
• a non-transitory computer-readable medium or media comprising one or more sequences of instructions which when executed by the one or more processors, causes steps to be performed comprising:
• receiving color and depth of a scene from a plurality of vantage points from one or more input devices;
• producing, for each vantage point, a three-dimensional (3D) model of data by combining depth data information and visible image data;
• defining a source-independent canonical space which is used to register two or more 3D models into that space;
• and
• performing scene registration to acquire geometrical transformations to merge two or more 3D models of the same scene from their corresponding different vantage points into a combined 3D model;
• wherein the scene registration includes a step of performing scene reregistration to improve one or more geometric transformations;
• wherein scene reregistration comprises the step of:
• [a] selecting as a basis a point cloud generated from data of the scene taken from a vantage point;
• [b] reprojecting point clouds generated from data of the scene taken from different vantage points onto the basis and treating the aggregation of the resulting combined points clouds as one virtual point cloud;
• [c] solving for rigid body movement and producing rotation (R*) and translation (T*) values;
• [d] using the produced rotation (R*) and translation (T*) values to update scene registration information;
• [e] responsive to a stop condition being reached, selecting a best set of scene registration information based on one or more evaluation metrics;
• and
• [f] responsive to a stop condition not being reached, iterating by returning to step [a].