# US 2026/0268596 A1 · Real time digital twin system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0268596 A1
- Document ID: US20260268596A1
- Status: Published application
- Kind: A1
- Application number: 19/165,566
- Filed: 2024-06-27
- Published: 2026-09-10
- Earliest claimed: 2023-06-27
- Assignees: Interaptix Inc.
- Applicants: Interaptix Inc.
- Inventors: Dae Hyun LEE; Bardia BINA; Luke Andrew ELLIOTT
- Web page: https://patentgrep.com/patent/US20260268596A1

## CPC

- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 7/12: Edge-based segmentation
- G06T 7/248: {involving reference images or patches}
- G06T 7/337: {involving reference images or patches}
- G06T 7/74: {involving reference images or patches}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2219/2008: Assembling, disassembling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present specification provides a method and system for computing a digital twin of a subject, which can enhance remote inspection and information retrieval. This system uses multiple fixed and movable capture devices, such as headsets and drones, to continuously collect image frames and telemetry data. The collected data is processed by a digital twin engine to create and update a comprehensive, real-time, or periodically updated three-dimensional digital twin. This method improves over traditional video feeds by enabling full 3D navigation and inspection of spaces like rooms, buildings, and vehicles.

## Claims

### Claim 1

A method of computing a digital twin, the method comprising:
capturing, at a plurality of capture devices (108), image frames representing the subject at respective scanning times;
establishing a root in at least one of the image frames, and estimating pose information for each image frame, relative to the root;
transmitting the image frames from the plurality of capture devices (108) to a digital twin engine (104);
aligning the image frames with existing frames in the digital twin based on feature matching and the pose estimation;
building an intermediate digital twin, at the digital twin engine (104), based on the alignment of the image frames; and
aligning image frames, at the digital twin engine (104), according to the respective scanning times to build the final digital twin.

### Claim 2 (depends on 1)

The method of claim 1, wherein the method is performed continuously or periodically to update the digital twin.

### Claim 3 (depends on 1)

The method of claim 1, wherein building the intermediate digital twin comprises:
accumulating image frames from the plurality of capture devices (108);
defining a boundary for the subject;
dividing the subject into segments; and
building the intermediate digital twin by assembling the segments.

### Claim 4 (depends on 1)

The method of claim 1 wherein the plurality of capture devices includes both fixed capture devices and movable capture devices, the method further comprising:
monitoring the subject by capturing image frames at one of the fixed capture devices and comparing the image frames to the existing frames;
determining whether a non-transient change has occurred in the subject based on the comparison;
if no non-transient changes have occurred, continuing to monitor the subject;
if a non-transient change has occurred, identifying a segment of the digital twin associated with the image frame as invalid;
controlling one of the movable capture devices to capture replacement image frames of the invalid segment; and
updating the digital twin based with the replacement image frames.

### Claim 5 (depends on 1)

The method of claim 1 further comprising:
estimating pose information for the image frames;
selecting reference images from the existing images;
estimating pose information for the reference images;
comparing the pose information of the image frames with the pose information of the reference images; and
correcting the pose information for the image frames based on the comparison.

### Claim 6 (depends on 1)

The method of claim 1, wherein the plurality of capture devices (108) includes a first capture device (108) and a second capture device (108), and the image frames captured by the first and second capture devices are processed according to the processing capacities of the respective capture devices.

### Claim 7 (depends on 6)

The method of claim 6, wherein the image frames comprise video frames and the first capture device further collects inertial measurement unit (IMU) data, the method further comprising:
estimating the pose information for each image frame at the first capture device (108) using a simultaneous localization and mapping (SLAM) algorithm;
wherein aligning the image frames with the existing frames is performed by the first capture device (108).

### Claim 8 (depends on 6)

The method of claim 6, wherein the image frames comprise video frames, wherein aligning the image frames with the existing frames comprises executing a structure from motion (SfM) algorithm at the digital twin engine (104).

### Claim 9 (depends on 6)

The method of claim 6, wherein:
the first capture device (108) is movable and includes an IMU sensor to detect
when the first capture device (108) is currently stationary, and
capturing the image frames is performed only while the first capture device (108) is stationary.

### Claim 10 (depends on 6)

The method of claim 6, wherein the image frames include photos, the method further comprising:
estimating the pose information for each image frame at the first capture device (108) using an ARKit™ application programming interface and a simultaneous localization and mapping (SLAM) algorithm.

### Claim 11

A system for computing a digital model of a subject, the system comprising:
a plurality of capture devices (108) configured to generate image frames representing a portion of the subject at respective scanning times;
a digital twin engine (104) configured to:
receive the image frames from the plurality of capture devices via a network;
establish a root in at least one of the image frames;
align the image frames with existing frames in the digital twin based on the root;
build an intermediate digital twin based on the alignment of the image frames; and
align the image frames with the respective scanning times to construct the final digital twin; and
a retrieval computing device (116) for outputting the digital twin.

### Claim 12 (depends on 11)

The system of claim 11, the plurality of capture devices including at least a first capture device configured to establish the root and align the image frames with the existing frames.

### Claim 13 (depends on 11)

The system of claim 11, wherein the plurality of capture devices (108) includes one or more of a video camera, a video phone, a digital camera, a headset, a tablet computer, and a drone.

### Claim 14 (depends on 13)

The system of claim 13, wherein the plurality of capture devices (108) includes both fixed capture devices and movable capture devices.

### Claim 15 (depends on 11)

The system of claim 11, wherein the retrieval computing device (116) is selected from the group consisting of a virtual reality device, a personal computing device, a smartphone, a tablet computer, and an augmented reality device.

### Claim 16 (depends on 11)

The system of claim 11 wherein the plurality of capture devices (108) includes at least a first capture device including a sensor for capturing telemetry data.