# US 12,223,748 B2 · Systems and methods for identifying changes in a space > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,748 B2 - Document ID: US12223748B2 - Status: Granted - Kind: B2 - Application number: 18/123,006 - Filed: 2023-03-17 - Published: 2023-09-21 - Granted: 2025-02-11 - Earliest claimed: 2022-03-18 - Assignees: Interaptix Inc. - Applicants: Interaptix Inc. - Inventors: Isaac Louis Gold Berman; Bardia Bina - Web page: https://patentgrep.com/patent/US12223748B2 ## CPC - G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking - G06V 20/64: Three-dimensional [3D] objects - G06T 3/18: Image warping, e.g. rearranging pixels individually - G06T 7/13: Edge detection - G06T 7/254: involving subtraction of images - G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration - G06T 7/50: Depth or shape recovery - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 19/003: {Navigation within 3D models or images} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06V 10/764: using classification, e.g. of video objects - G06V 20/50: Context or environment of the image - G06V 20/647: {by matching two-dimensional images to three-dimensional objects} - G06T 2200/04: involving 3D image data - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/10048: Infrared image - G06T 2210/04: Architectural design, interior design - G06T 2210/61: Scene description - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An example method includes: retrieving a previous representation representing a target portion of a space; obtaining a current representation representing the target portion of the space; comparing the previous representation to the current representation to identify a change in the target portion of the space; mapping the target portion of the space to a three-dimensional (3D) representation of the space to locate the change within the 3D representation; and storing a change map defining the change in context of the 3D representation. ## Claims ### Claim 1 A method comprising: retrieving a previous representation representing a target portion of a space; obtaining a current representation representing the target portion of the space; comparing the previous representation to the current representation to identify a change in the target portion of the space; mapping the target portion of the space to a three-dimensional (3D) representation of the space to locate the change within the 3D representation; and storing a change map defining the change in context of the 3D representation. ### Claim 2 (depends on 1) The method of claim 1, wherein the previous representation and the current representation comprise images. ### Claim 3 (depends on 2) The method of claim 2, wherein comparing the previous representation and the current representation to identify the change comprises: warping the images to align the previous representation and the current representation; computing a structural similarity between the previous representation and the current representation to obtain a plurality of potential changes; and for each potential change having a structural similarity value below a threshold similarity value, defining the potential change as a candidate change. ### Claim 4 (depends on 3) The method of claim 3, wherein comparing the previous representation and the current representation to identify the change further comprises: detecting edges in at least one of the current representation or the previous representation; and filtering out the potential changes corresponding to one of the detected edges. ### Claim 5 (depends on 3) The method of claim 3, wherein comparing the previous representation and the current representation to identify the change further comprises: combining potential changes within a threshold distance of one another. ### Claim 6 (depends on 1) The method of claim 1, wherein the previous representation and the current representation comprise 3D scans of the space, the 3D scans including image data and depth data. ### Claim 7 (depends on 6) The method of claim 6, wherein comparing the previous representation and the current representation to identify the change comprises: mapping the previous representation and the current representation to a common frame of reference; and comparing image data from the 3D scans to compute a structural similarity between the previous representation and the current representation to obtain a plurality of potential changes; and for each potential change having a structural similarity value below a threshold similarity value, defining the potential change as a candidate change. ### Claim 8 (depends on 1) The method of claim 1, wherein retrieving the previous representation comprises: presenting a plurality of previously captured representations of the target portion of the space; and selecting one of the previously captured representations as the previous representation based on input from a user. ### Claim 9 (depends on 1) The method of claim 1, further comprising: presenting a candidate change for confirmation at a client device; and in response to an affirmative input from a user, defining the candidate change as a confirmed change. ### Claim 10 (depends on 1) The method of claim 1, further comprising: obtaining previous depth data and current depth data representing the change; when the current depth data is closer than the previous depth data, classifying the change as an addition; and when the previous depth data is closer than the current depth data, classifying the change as a deletion. ### Claim 11 (depends on 10) The method of claim 10, wherein the change map comprises current depth data depicted in a first format when the change is classified as an addition and previous depth data depicted in a second format when the change is classified as a deletion. ### Claim 12 (depends on 1) The method of claim 1, wherein the change map comprises a layer associated with the 3D representation, the layer including an annotation identifying a location of the change. ### Claim 13 (depends on 12) The method of claim 12, wherein the annotation is mapped to a nearest underlying surface in the 3D representation. ### Claim 14 (depends on 1) The method of claim 1, further comprising displaying the change map at a client device, wherein the change map is presented as one of: a navigable 3D representation and augmented reality components overlaid in a current data capture view. ### Claim 15 (depends on 1) The method of claim 1, wherein the previous representation and the current representation comprise thermal images. ### Claim 16 A server comprising: a memory and a communications interface; and a processor interconnected with the memory and the communications interface, the processor configured to: retrieve, from the memory, a previous representation representing a target portion of a space; receive, via the communications interface, a current representation representing the target portion of the space; compare the previous representation to the current representation to identify a change in the target portion of the space; map the target portion of the space to a three-dimensional (3D) representation of the space to locate the change within the 3D representation; and store a change map defining the change in context of the 3D representation. ### Claim 17 (depends on 16) The server of claim 16, wherein the previous representation and the current representation comprise images, and wherein to compare the previous representation and the current representation, the processor is configured to: warp the images to align the previous representation and the current representation; compute a structural similarity between the previous representation and the current representation to obtain a plurality of potential changes; and for each potential change having a structural similarity value below a threshold similarity value, define the potential change as a candidate change. ### Claim 18 (depends on 17) The server of claim 17, wherein to compare the previous representation and the current representation, the processor is further configured to: detect edges in at least one of the current representation or the previous representation; and filter out the potential changes corresponding to one of the detected edges. ### Claim 19 (depends on 16) The server of claim 16, wherein the previous representation and the current representation comprise 3D scans of the space, the 3D scans including image data and depth data and wherein to compare the previous representation and the current representation to identify the changes, the processor is configured to: map the previous representation and the current representation to a common frame of reference; and compare image data from the 3D scans to compute a structural similarity between the previous representation and the current representation to obtain a plurality of potential changes; and for each potential change having a structural similarity value below a threshold similarity value, define the potential change as a candidate change. ### Claim 20 (depends on 16) The server of claim 16, wherein the processor is further configured to: obtain previous depth data and current depth data representing the change; when the current depth data is closer than the previous depth data, classify the change as an addition; and when the previous depth data is closer than the current depth data, classify the change as a deletion. ### Claim 21 A system comprising: a database storing previously captured representations of a space and a persistent 3D representation of the space; a data capture device configured to capture a current representation of the space; and a server configured to: retrieve, from the database, one of the previously captured representations; receive, from the data capture device, the current representation of the space; compare the previously captured representation and the current representation to identify a change in the space; locate the change within the persistent 3D representation; and generate a change map defining the change in context of the persistent 3D representation.