# US 12,223,613 B2 · Tracking an ongoing construction by using fiducial markers > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,613 B2 - Document ID: US12223613B2 - Status: Granted - Kind: B2 - Application number: 18/460,310 - Filed: 2023-09-01 - Published: 2024-07-25 - Granted: 2025-02-11 - Earliest claimed: 2018-11-09 - Assignees: Doxel, Inc. - Applicants: Doxel, Inc. - Inventors: Anton Treskunov; Robin Singh - Web page: https://patentgrep.com/patent/US12223613B2 ## CPC - G05D 1/0251: {extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593)} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G01S 17/89: for mapping or imaging - G05D 1/0236: {in combination with a laser (lasers per se H01S)} - G05D 1/024: {in combination with a laser (lasers per se H01S)} - G05D 1/247: using signals provided by artificial sources external to the vehicle, e.g. navigation beacons - G06F 30/12: characterised by design entry means specially adapted for CAD, e.g. graphical user interfaces [GUI] specially adapted for CAD - G06F 30/13: Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads - G06T 7/33: using feature-based methods - G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light - G06T 15/205: {Image-based rendering} - G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes - G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders - G06T 2207/10016: Video; Image sequence - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30208: Marker matrix - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for tracking progression of a construction site at different points in time relative to an expected progression of the construction site as represented by a computer model. The method includes generating an image of the construction site by scanning the construction site with an imaging device and detecting a fiducial marker appearing in the image at a location on the construction site. The fiducial marker is unique among fiducial markers positioned on the construction site. The method further includes identifying the detected fiducial marker in association with a unique identifier among the fiducial markers and aligning a coordinate system of the identified fiducial marker with a coordinate system of the computer model, which represents the construction site at points in time. ## Claims ### Claim 1 A system for tracking progression of a construction site at different points in time relative to an expected progression of the construction site as represented by a computer model, the system comprising: one or more processor; and one or more memories including instructions executable by the processors causing the system to: generate a first plurality of two-dimensional (2D) images of the construction site by scanning the construction site with an imaging device; detect a first set of fiducial markers appearing in the first plurality of 2D images, each of the first set of fiducial markers being unique among the first set of fiducial markers positioned on the construction site; generate a second plurality of 2D images of the construction site; detect a second set of fiducial markers appearing in the second plurality of 2D images, each of the second set of fiducial markers being unique among the second set of fiducial markers; identify each of the second set of fiducial markers in association with a unique identifier and a unique location relative to the construction site; and align a coordinate system of the second set of fiducial markers with a coordinate system of the computer model utilizing a transformation function based on the first set of fiducial markers, which represents the construction site at points in time. ### Claim 2 (depends on 1) The system of claim 1, wherein each of the fiducial markers in the first set of fiducial markers appears in the first plurality of 2D images, and each of the fiducial markers in the second set of fiducial markers appears in the second plurality of 2D images captured of the construction site. ### Claim 3 (depends on 1) The system of claim 1, wherein the construction site includes a plurality of three-dimensional (3D) physical elements. ### Claim 4 (depends on 1) The system of claim 1, wherein the construction site is a partially complete construction of a three-dimensional (3D) structure. ### Claim 5 (depends on 1) The system of claim 1 further comprising: a robotic unmanned vehicle including a light detection and ranging (LIDAR) system of the imaging device, wherein the robotic unmanned vehicle is configured to traverse the construction site to capture the first and second pluralities of 2D images. ### Claim 6 (depends on 1) The system of claim 1, wherein the first set of fiducial markers is positioned such that the first plurality of 2D images of the construction site includes an image of each of the first set of fiducial markers, and wherein the second set of fiducial markers is positioned such that the second plurality of 2D images of the construction site includes an image of each of the second set of fiducial markers. ### Claim 7 (depends on 1) The system of claim 1, wherein each of the first set of fiducial markers and each of the second set of fiducial markers is positioned on a surface of a physical element of the construction site. ### Claim 8 (depends on 1) The system of claim 1, wherein each of the first set of fiducial markers and each of the second set of fiducial markers is a two-dimensional (2D) barcode that is unique relative to every fiducial marker. ### Claim 9 (depends on 1) The system of claim 1, wherein each of the first set of fiducial markers and each of the second set of fiducial markers has a unique grayscale or binary pattern on a planar surface. ### Claim 10 A non-transitory computer-readable storage medium that stores instructions to be executed by at least one processor of a system, wherein the instructions cause the system to: generate a first plurality of two-dimensional (2D) images of a construction site by scanning the construction site with an imaging device; detect a first set of fiducial markers appearing in the first plurality of 2D images, each of the first set of fiducial markers being unique among the first set of fiducial markers positioned on the construction site; generate a second plurality of 2D images of the construction site; detect a second set of fiducial markers appearing in the second plurality of 2D images, each of the second set of fiducial markers being unique among the second set of fiducial markers; identify each of the second set of fiducial markers in association with a unique identifier and a unique location relative to the construction site; and align a coordinate system of the second set of fiducial markers with a coordinate system of a computer model utilizing a transformation function based on the first set of fiducial markers, which represents the construction site at points in time. ### Claim 11 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein each of the fiducial markers in the first set of fiducial markers appears in the first plurality of 2D images, and each of the fiducial markers in the second set of fiducial markers appears in the second plurality of 2D images captured of the construction site. ### Claim 12 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the construction site includes a plurality of three-dimensional (3D) physical elements. ### Claim 13 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the construction site is a partially complete construction of a three-dimensional (3D) structure. ### Claim 14 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the system is further caused to: cause a robotic unmanned vehicle including a light detection and ranging (LIDAR) system of the imaging device, to traverse the construction site to capture the first and second pluralities of 2D images. ### Claim 15 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the first set of fiducial markers is positioned such that the first plurality of 2D images of the construction site includes an image of each of the first set of fiducial markers, and wherein the second set of fiducial markers is positioned such that the second plurality of 2D images of the construction site includes an image of each for the second set of fiducial markers. ### Claim 16 A method tracking progression of a construction site at different points in time relative to an expected progression of the construction site as represented by a computer model, the method comprising: generating a first plurality of two-dimensional (2D) images of the construction site by scanning the construction site with an imaging device; detecting a first set of fiducial markers appearing in the first plurality of 2D images, each of the first set of fiducial markers being unique among the first set of fiducial markers positioned on the construction site; generating a second plurality of 2D images of the construction site; detecting a second set of fiducial markers appearing in the second plurality of 2D images, each of the second set of fiducial markers being unique among the second set of fiducial markers; identifying each of the second set of fiducial markers in association with a unique identifier and a unique location relative to the construction site; and aligning a coordinate system of the second set of fiducial markers with a coordinate system of the computer model utilizing a transformation function based on the first set of fiducial markers, which represents the construction site at points in time. ### Claim 17 (depends on 16) The method of claim 16, wherein the first set of fiducial markers is positioned such that the first plurality of 2D images of the construction site includes an image of each of the first set of fiducial markers, and wherein the second set of fiducial markers is positioned such that the second plurality of 2D images of the construction site includes an image of each of the second set of fiducial markers. ### Claim 18 (depends on 16) The method of claim 16, wherein each of the first set of fiducial markers and each of the second set of fiducial markers is positioned on a surface of a physical element of the construction site. ### Claim 19 (depends on 16) The method of claim 16, wherein each of the first set of fiducial markers and each of the second set of fiducial markers is a two-dimensional (2D) barcode that is unique relative to every fiducial marker. ### Claim 20 (depends on 16) The method of claim 16, wherein each of the first set of fiducial markers and each of the second set of fiducial markers has a unique grayscale or binary pattern on a planar surface.