# US 12,182,934 B2 · Four-dimensional data platform using automatic registration for different data sources > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,934 B2 - Document ID: US12182934B2 - Status: Granted - Kind: B2 - Application number: 17/967,221 - Filed: 2022-10-17 - Published: 2023-04-20 - Granted: 2024-12-31 - Earliest claimed: 2021-10-18 - Applicants: FARO Technologies, Inc. - Inventors: Oliver Zweigle; Thorsten Brecht - Web page: https://patentgrep.com/patent/US12182934B2 ## CPC - G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar - G06T 7/337: {involving reference images or patches} - G06T 7/74: {involving reference images or patches} - G06T 17/005: {Tree description, e.g. octree, quadtree} - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30184: Infrastructure - G06T 2210/56: Particle system, point based geometry or rendering - G06T 2210/61: Scene description - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES ## Abstract A method is provided that includes generating a four-dimensional (4D) model of an environment based on three-dimensional (3D) coordinates of the environment captured at a first point in time. The method further includes updating the 4D model based at least in part to an update to at least a subset of the 3D coordinates of the environment captured at a second point in time. The method further includes enriching the 4D model by adding supplemental information to the model. ## Claims ### Claim 1 A method comprising: generating a four-dimensional (4D) model of an environment based on three-dimensional (3D) coordinates of the environment captured at a first point in time corresponding to a first timestamp; updating the 4D model based at least in part-te on an update to at least a subset of the 3D coordinates of the environment captured at a second point in time corresponding to a second timestamp; and enriching the 4D model by adding supplemental information comprising at least one of text entered by a user, an image from a camera that is localized on the 4D model, and data from a device; removing at least one of a stray point, a transient object, and a reflection data based on timestamps; storing the 4D model in a first storage location; archiving a subset of the 4D model based on access thereof over time and a timestamp relating to the subset, the archiving being to a second storage location having an access rate that is slower than the first storage location, the subset of the 4D model including at least one of 3D data and metadata. ### Claim 2 (depends on 1) The method of claim 1, wherein generating the 4D model comprises performing at least one of feature detection and matching, object recognition and matching, and pairwise registration. ### Claim 3 (depends on 2) The method of claim 2, wherein the feature detection and matching is performed using a random sample consensus approach to identify feature matches. ### Claim 4 (depends on 2) The method of claim 2, wherein the pairwise registration is performed using a two-dimensional approach to collapse the 3D coordinates into a pre-defined plane. ### Claim 5 (depends on 2) The method of claim 2, wherein the pairwise registration is performed using a cloud2cloud algorithm. ### Claim 6 (depends on 1) The method of claim 1, wherein the generating of the 4D model comprises performing post processing on the 3D coordinates. ### Claim 7 (depends on 6) The method of claim 6, wherein the post processing comprises: at least one of moving an object and performing denoising; and performing the removing of at least one of a stray point, a transient object, and a reflection data based on a timestamp. ### Claim 8 (depends on 1) The method of claim 1, wherein the 3D coordinates are obtained from at least one of (i) a laser scanner, (ii) a lidar sensor of a smartphone or mobile scanner, and (iii) an imaging device-via in conjunction with a photogrammetry technique. ### Claim 9 (depends on 1) The method of claim 1, wherein updating the model comprises registering the at least the subset of the 3D coordinates of the environment to the 4D model. ### Claim 10 (depends on 1) The method of claim 1, wherein a timestamp is associated with the supplemental information. ### Claim 11 (depends on 1) The method of claim 1, wherein enriching the 4D model comprises localizing the supplemental information to the 4D model. ### Claim 12 (depends on 1) The method of claim 1, further comprising processing and analyzing the 4D model. ### Claim 13 A system comprising: a memory comprising computer readable instructions; and a processing device for executing the computer readable instructions, the computer readable instructions controlling the processing device to perform operations comprising: generating a four-dimensional (4D) model of an environment based on three-dimensional (3D) coordinates of the environment captured at a first point in time; updating the 4D model based at least in part to an update to at least a subset of the 3D coordinates of the environment captured at a second point in time; and enriching the 4D model by adding supplemental information comprising text, an image, or data from a device to the model; removing at least one of a stray point, a transient object, and a reflection data based on timestamps; storing the 4D model in a first storage location; archiving a subset of the 4D model based on spatial and temporal characteristics, the archiving being to a second storage location, the second storage location having an access rate that is slower than the first storage location, the subset of the 4D model including at least one of 3D data and metadata. ### Claim 14 (depends on 13) The system of claim 13, further comprising a 3D coordinate measurement device to capture the 3D coordinates of the environment. ### Claim 15 (depends on 14) The system of claim 14, wherein the 3D coordinate measurement device is a laser scanner. ### Claim 16 (depends on 13) The system of claim 13, wherein the supplemental information is captured by a camera. ### Claim 17 A method comprising: localizing a sensor within a physical environment using at least one of text, an image, and data from a device; registering a dataset in model of the physical environment; generating a four-dimensional (4D) model of the physical environment based on the dataset and timestamps of data captured in the dataset; removing at least one of a stray point, a transient object, and a reflection data based on timestamps; merging different qualities of data within the dataset and overwriting corresponding elements with lower quality data; maintaining dimension of time in dataset as a timestamp representing the time of collection of the data; storing the 4D model in a first storage location; and archiving a subset of the 4D model based on access thereof over time and a timestamp relating to the subset, the archiving being to a second storage location having an access rate that is slower than the first storage location, the subset of the 4D model including at least one of 3D data and metadata. ### Claim 18 (depends on 17) The method of claim 17, wherein the localizing of the sensor further comprises comparing and matching features of an existing point cloud with a new scan and determining a position in 3D space.