4D vizualization of building design and construction modeling with photographs

A system comprising:

• at least one processing device;
• non-transitory computer-readable medium storing instructions and a three-dimensional (“3D”) point cloud model of building progress of a construction site comprising time-lapsed pixels obtained from a plurality of images, the at least one processing device to execute the instructions to:
• identify a structure within the 3D point cloud model that does not occupy the same pixels in a majority of the time-lapsed pixels over time;
• determine a background image within the 3D point cloud model by computing a per-pixel median of the time-lapsed pixels;
• compute a squared pixel-wise difference between the structure and the background image, to identify a plurality of occluding pixels of the structure;
• and
• smooth the plurality of occluding pixels with a cross-bilateral filter, to create a smoothed occluded image removable from the 3D point cloud model.

A system comprising:

• at least one processing device;
• non-transitory computer-readable medium storing instructions and a three-dimensional (“3D”) point cloud model of building progress of a construction site comprising pixels obtained from a plurality of images, the at least one processing device to execute the instructions to:
• for each of a plurality of points within the 3D point cloud model projected from a building information model (BIM) to corresponding points in one of the plurality of images, execute a heuristic on the point comprising:
• [p−pmodel>0.3]v[cos−1(n(p)Tn(pmodel))>π/6]
• wherein pmodel is a location of the point on a mesh of the BIM to which the point projects and n(p) calculates a surface normal at the point, to generate estimates of sparse occlusion pixels corresponding to static occlusions;
• flood superpixels of the 3D point cloud model with the estimates of the sparse occlusion pixels;
• identify a plurality of the superpixels, each containing at least one of the sparse occlusion pixels;
• and
• group the plurality of identified superpixels into an occlusion mask selectably removable from a displayed rendering of the construction site.

A non-transitory computer-readable medium having instructions encoded thereon that, when executed by a processing device, cause the processing device to:

• receive an anchor image chosen from a first plurality of images taken of a construction site;
• receive indications of a plurality of first points in the anchor image that correspond to a plurality of second points in a 3D building information model (BIM);
• calculate camera calibration information for an anchor camera from which the anchor image was taken based on the indications and on metadata extracted from the anchor image, to register the anchor image in relation to the BIM, and thus generate a registered anchor image;
• determine a homography transformation between the first plurality of images and the anchor camera using the camera calibration information for the anchor camera, to register a second plurality of images selected from the first plurality of images other than the anchor image, wherein some of the second plurality of images come from at least one additional camera and from an angle different than that of the anchor camera;
• and
• construct a visualization of progress of a construction project in relation to the BIM based on the registered anchor image and the registered second plurality of images.

A method comprising:

• storing, by a processing device, a three-dimensional (“3D”) point cloud model of building progress of a construction site comprising time-lapsed pixels obtained from a plurality of images;
• identifying, by a processing device, a structure within the 3D point cloud model that does not occupy the same pixels in a majority of the time-lapsed pixels over time;
• determining, by the processing device, a background image within the 3D point cloud model by computing a per-pixel median of the time-lapsed pixels;
• computing, by the processing device, a squared pixel-wise difference between the structure and the background image, to identify a plurality of occluding pixels of the structure;
• and
• smoothing, by the processing device, the plurality of occluding pixels with a cross-bilateral filter, to create a smoothed occluded image that is removable from the 3D point cloud model.

A method comprising:

• storing, by a processing device, a three-dimensional (“3D”) point cloud model of building progress of a construction site comprising pixels obtained from a plurality of images;
• for each of a plurality of points within the 3D point cloud model projected from a building information model (BIM) to corresponding points in one of the plurality of images:
• executing, by the processing device, a heuristic on the point comprising:
• [p−pmodel >0.3]V [cos−1(n(p)Tn(pmodel)) > π/6], wherein pmodel is a location of the point on a mesh of the BIM to which the point projects and n(p) calculates a surface normal at the point, to generate estimates of sparse occlusion pixels corresponding to static occlusions;
• flooding, by the processing device, superpixels of the 3D point cloud model with the estimates of the sparse occlusion pixels;
• identifying, by the processing device, a plurality of the superpixels, each containing at least one of the sparse occlusion pixels;
• and
• grouping, by the processing device, the plurality of identified superpixels into an occlusion mask that is selectably removable from a displayed rendering of the construction site.