# US 12,217,380 B2 · 3-D reconstruction using augmented reality frameworks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,380 B2 - Document ID: US12217380B2 - Status: Granted - Kind: B2 - Application number: 18/331,896 - Filed: 2023-06-08 - Published: 2023-10-05 - Granted: 2025-02-04 - Earliest claimed: 2019-12-13 - Assignees: Hover Inc. - Applicants: Hover, Inc. - Inventors: Manish Upendran; William Castillo; Jena Dzitsiuk; Yunwen Zhou; Matthew Thomas; Giridhar Murali; Atulya Shree - Web page: https://patentgrep.com/patent/US12217380B2 ## CPC - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T 7/74: {involving reference images or patches} - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2210/04: Architectural design, interior design - G06T 2219/2016: Rotation, translation, scaling - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract System and method are provided for scaling a 3-D representation of a building structure. The method includes obtaining world map data including a first track of real-world poses for a plurality of images. The plurality of images comprises non-camera anchors. The method also includes detecting a discrepancy in at least one real-world pose of the first track. The method also includes in response to detecting a discrepancy, generating a new track of real-world poses. The method also includes calculating a scaling factor for a 3-D representation of the building structure based on sampling across a plurality of tracks. The plurality of tracks comprises at least the first track and the new track. ## Claims ### Claim 1 A method for validating camera information in 3-D reconstruction of a building structure, the method comprising: obtaining world map data including a first track of real-world poses for a plurality of images, wherein the plurality of images comprises non-camera anchors; detecting a discrepancy in at least one real-world pose of the first track; in response to detecting a discrepancy, generating a new track of real-world poses; and calculating a scaling factor for a 3-D representation of the building structure based on sampling across a plurality of tracks, wherein the plurality of tracks comprises at least the first track and the new track. ### Claim 2 (depends on 1) The method of claim 1, wherein detecting the discrepancy comprises: obtaining a plurality of images of the building structure, wherein the plurality of images comprises non-camera anchors; generating a reference pose track comprising estimated camera poses based on feature matching of non-camera anchors; and determining if at least one estimated pose of the reference pose track and a corresponding real-world poses of the first track are separated by more than a first predetermined threshold distance. ### Claim 3 (depends on 2) The method of claim 2, wherein determining if the at least one estimated pose of the reference pose track and the real-world poses of the first track are separated by more than the first predetermined threshold distance is performed after predetermined time intervals. ### Claim 4 (depends on 2) The method of claim 2, wherein determining if the at least one estimated pose of the reference pose track and the real-world poses of the first track are separated by more than the first predetermined threshold distance is performed based on determining if tracking quality for the world map data is below a predetermined threshold. ### Claim 5 (depends on 2) The method of claim 2, wherein determining if the at least one estimated pose of the reference pose track and the real-world poses of the first track are separated by more than the first predetermined threshold distance is performed based on detecting if a device used to capture the plurality of images is moved by more than a predetermined threshold distance. ### Claim 6 (depends on 1) The method of claim 1, wherein detecting the discrepancy comprises determining if tracking quality for the world map data is below a predetermined threshold. ### Claim 7 (depends on 1) The method of claim 1, wherein generating the new track of real-world poses comprises resetting an Augmented Reality (AR) program of a device used to obtain the world map data. ### Claim 8 (depends on 7) The method of claim 7, wherein resetting the AR program comprises resetting an Inertial Measurement Unit (IMU) of the device. ### Claim 9 (depends on 1) The method of claim 1, wherein generating the new track of real-world poses comprises generating a new reference pose track using cumulative data points for feature matching. ### Claim 10 (depends on 1) The method of claim 1, wherein generating the new track of real-world poses comprises guiding a user of a device used to capture the plurality of images to a last location when the estimated poses of the reference pose track and the real-world poses of the first track were consistent. ### Claim 11 (depends on 1) The method of claim 1, wherein the sampling is biased to use a middle portion of each track of the plurality of tracks. ### Claim 12 (depends on 1) The method of claim 1, further comprising: weighting one or more tracks of the plurality tracks higher than other tracks that are longer, while sampling the plurality of tracks. ### Claim 13 (depends on 1) The method of claim 1, further comprising: weighting one or more tracks of the plurality tracks higher than other tracks with associated IMU drifts, while sampling the plurality of tracks. ### Claim 14 (depends on 1) The method of claim 1, further comprising: weighting one or more tracks of the plurality tracks with more than one planar surface higher than other tracks, while sampling the plurality of tracks. ### Claim 15 (depends on 1) The method of claim 1, further comprising: weighting one or more tracks of the plurality tracks higher than other tracks based on a tracking state of an AR framework used to obtain the world map data, while sampling the plurality of tracks. ### Claim 16 A computer system for 3-D reconstruction of a building structure, comprising: one or more processors, including a general purpose processor and a graphics processing unit (GPU); a display; and memory; wherein the memory stores one or more programs configured for execution by the one or more processors, and the one or more programs comprising instructions for: obtaining world map data including a first track of real-world poses for a plurality of images, wherein the plurality of images comprises non-camera anchors; detecting a discrepancy in at least one real-world pose of the first track; in response to detecting a discrepancy, generating a new track of real-world poses; and calculating a scaling factor for a 3-D representation of the building structure based on sampling across a plurality of tracks, wherein the plurality of tracks comprises at least the first track and the new track. ### Claim 17 (depends on 16) The computer system of claim 16, wherein detecting the discrepancy comprises: obtaining a plurality of images of the building structure, wherein the plurality of images comprises non-camera anchors; generating a reference pose track comprising estimated camera poses based on feature matching of non-camera anchors; and determining if at least one estimated pose of the reference pose track and a corresponding real-world poses of the first track are separated by more than a first predetermined threshold distance. ### Claim 18 (depends on 16) The computer system of claim 16, wherein detecting the discrepancy comprises determining if tracking quality for the world map data is below a predetermined threshold. ### Claim 19 (depends on 17) The computer system of claim 17, wherein determining if the at least one estimated pose of the reference pose track and the real-world poses of the first track are separated by more than the first predetermined threshold distance is performed after predetermined time intervals. ### Claim 20 A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer system having a display, one or more processors including a general purpose processor and a graphical processing unit (GPU), the one or more programs comprising instructions for: obtaining world map data including a first track of real-world poses for a plurality of images, wherein the plurality of images comprises non-camera anchors; detecting a discrepancy in at least one real-world pose of the first track; in response to detecting a discrepancy, generating a new track of real-world poses; and calculating a scaling factor for a 3-D representation of the building structure based on sampling across a plurality of tracks, wherein the plurality of tracks comprises at least the first track and the new track.