# US 2026/0011151 A1 · Systems and methods for mixed reality (mr) and artificial intelligence (ai)-enhanced fire investigation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0011151 A1
- Document ID: US20260011151A1
- Status: Published application
- Kind: A1
- Application number: 19/326,614
- Filed: 2025-09-11
- Published: 2026-01-08
- Earliest claimed: 2023-11-07
- Assignees: The Travelers Indemnity Company
- Applicants: The Travelers Indemnity Company
- Inventors: Ryan M. Scanlon; Douglas L. Roy; Reyer D. Swengel; Ryan M. Malone; Pee T. Lim; Keira R. Skenandore
- Web page: https://patentgrep.com/patent/US20260011151A1

## CPC

- G06V 20/50: Context or environment of the image
- G06T 7/73: using feature-based methods
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88)
- G06V 20/20: in augmented reality scenes
- G08B 17/125: {by using a video camera to detect fire or smoke}
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20081: Training; Learning
- G06T 2207/30196: Human being; Person
- G06T 2207/30244: Camera pose
- G06T 2210/56: Particle system, point based geometry or rendering
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS

## Abstract

A Mixed Reality (MR) and Artificial Intelligence (AI)-enhanced fire investigation system and method that analyzes data descriptive of fire-damaged locations, identifies objects of the location, creates a 3-D model of the location, automatically analyzes the location utilizing an AI fire investigation model, automatically analyzes the location utilizing an AI safety evaluation model, and automatically embeds and provides layered access to data assigned to the 3-D model.

## Claims

### Claim 1

A method for Mixed Reality (MR) and Artificial Intelligence (AI)-enhanced fire investigation, comprising:
acquiring, by a Time of Flight (ToF) sensor of an MR device, the ToF sensor being in communication with a processing device of the MR device, and at a first time and from a first location in an environment in which a wearer of a head-mounted see-through display of the MR device is located, the head-mounted see-through display being in communication with the processing device, data descriptive of first distances from the MR device to a first plurality of surface points in the environment, wherein the first plurality of surface points are within a first field of view of a camera of the MR device, the camera being in communication with the processing device;
computing, by the processing device and utilizing the first distances and the first location, a first portion of a 3-D point cloud descriptive of locations of the first plurality of surface points in the environment;
tracking, after the acquiring of the first distances and by one or more Inertial Measurement Unit (IMU) devices of the MR device, the one or more IMU devices being in communication with the processing device, a first movement of the wearer from the first location in the environment to a second location in the environment;
acquiring, by the ToF sensor and at a second time and from the second location in the environment, data descriptive of second distances from the MR device to a second plurality of surface points in the environment, wherein the second plurality of surface points are within a second field of view of the camera;
computing, by the processing device and utilizing the second distances and the second location, a second portion of the 3-D point cloud descriptive of locations of the second plurality of surface points in the environment;
generating, by the processing device and utilizing the first and second portions of the 3-D point cloud, a 3-D wire mesh model descriptive of the environment;
receiving, by a wireless communication device of the MR device, the wireless communication device being in communication with the processing device, and from a sensor device in selective communication with the wireless communication device, (i) data descriptive of the environment that has been captured by the sensor device and (ii) positioning information descriptive of a location and orientation of the sensor device;
identifying, by the processing device and based on the positioning information descriptive of a location and orientation of the sensor device, a portion of the 3-D wire mesh model that corresponds to the data descriptive of the environment that has been captured by the sensor device; and
assigning, by the processing device and to the corresponding portion of the 3-D wire mesh model, an attribute representative of the data descriptive of the environment that has been captured by the sensor device.

### Claim 2 (depends on 1)

The method for MR and AI-enhanced fire investigation of claim 1, wherein the sensor device comprises a camera and wherein the data descriptive of the environment that has been captured by the sensor device comprises one or more images.

### Claim 3 (depends on 2)

The method for MR and AI-enhanced fire investigation of claim 2, further comprising:
projecting, by the processing device, the one or more images onto the corresponding portion of the 3-D wire mesh model.

### Claim 4 (depends on 2)

The method for MR and AI-enhanced fire investigation of claim 2, further comprising:
identifying, by the processing device and for each portion of the one or more images, a corresponding portion of the 3-D wire mesh model; and
assigning, by the processing device and to each corresponding portion of the 3-D wire mesh model, a texture attribute representative of the corresponding portion of the one or more images.

### Claim 5 (depends on 1)

The method for MR and AI-enhanced fire investigation of claim 1, wherein the sensor device comprises a DSLR camera and wherein the data descriptive of the environment that has been captured by the sensor device comprises one or more high-resolution images.

### Claim 6 (depends on 5)

The method for MR and AI-enhanced fire investigation of claim 5, further comprising:
identifying, by the processing device and for each portion of the one or more high-resolution images, a corresponding portion of the 3-D wire mesh model; and
assigning, by the processing device and to each corresponding portion of the 3-D wire mesh model, an attribute representative of the corresponding portion of the one or more high-resolution images.

### Claim 7 (depends on 6)

The method for MR and AI-enhanced fire investigation of claim 6, further comprising:
outputting, via the head-mounted see-through display and to the wearer, an MR element that is indicative of the one or more high-resolution images, wherein the MR element is output such that it appears to be positioned in the environment at a location of the corresponding portion of the 3-D wire mesh model.

### Claim 8 (depends on 7)

The method for MR and AI-enhanced fire investigation of claim 7, further comprising:
receiving, from the wearer, an indication of a selection of the MR element; and
outputting, to the wearer and in response to the receiving, the one or more high-resolution images.

### Claim 9 (depends on 2)

The method for MR and AI-enhanced fire investigation of claim 2, wherein the environment comprises an interior of a fire-damaged structure and wherein the one or more images comprises imagery descriptive of fire damage.

### Claim 10 (depends on 1)

The method for MR and AI-enhanced fire investigation of claim 1, wherein the sensor device comprises a LiDAR device and wherein the data descriptive of the environment that has been captured by the sensor device comprises one or more distance measurements.

### Claim 11 (depends on 10)

The method for MR and AI-enhanced fire investigation of claim 10, further comprising:
identifying, by the processing device and for each portion of the one or more distance measurements, a corresponding portion of the 3-D wire mesh model;
computing, by the processing device, a difference between (i) a distance between the portion of the 3-D wire mesh model and the location of the sensor LiDAR device and (ii) the one or more distance measurements; and
computing, by the processing device and based on the difference, an error metric for the location of the corresponding portion of the 3-D wire mesh model.

### Claim 12 (depends on 1)

The method for MR and AI-enhanced fire investigation of claim 1, wherein the MR device comprises a battery in communication with at least one of the processing device, the ToF sensor, the head-mounted see-through display, the camera, the one or more IMU devices, and the wireless communication device.

### Claim 13

A method for Mixed Reality (MR) and Artificial Intelligence (AI)-enhanced fire investigation, comprising:
acquiring, by a Time of Flight (ToF) sensor of an MR device, the ToF sensor being in communication with a processing device of the MR device, and at a first time and from a first location in an environment in which a wearer of a head-mounted see-through display of the MR device is located, the head-mounted see-through display being in communication with the processing device, data descriptive of first distances from the MR device to a first plurality of surface points in the environment, wherein the first plurality of surface points are within a first field of view of a camera of the MR device, the camera being in communication with the processing device;
computing, by the processing device and utilizing the first distances and the first location, a first portion of a 3-D point cloud descriptive of locations of the first plurality of surface points in the environment;
tracking, after the acquiring of the first distances and by one or more Inertial Measurement Unit (IMU) devices of the MR device, the one or more IMU devices being in communication with the processing device, a first movement of the wearer from the first location in the environment to a second location in the environment;
acquiring, by the ToF sensor and at a second time and from the second location in the environment, data descriptive of second distances from the MR device to a second plurality of surface points in the environment, wherein the second plurality of surface points are within a second field of view of the camera;
computing, by the processing device and utilizing the second distances and the second location, a second portion of the 3-D point cloud descriptive of locations of the second plurality of surface points in the environment;
generating, by the processing device and utilizing the first and second portions of the 3-D point cloud, a 3-D wire mesh model descriptive of the environment;
identifying, by the processing device and by an execution of an AI safety criteria model stored in a memory device of the MR device, the memory device being in communication with the processing device, a safety hazard in the environment; and
outputting, via the head-mounted see-through display and to the wearer, an MR element that is indicative of the safety hazard.

### Claim 14 (depends on 13)

The method for MR and AI-enhanced fire investigation of claim 13, wherein the MR element comprises a prompt requesting input from the wearer, the method further comprising:
identifying, by the processing device, an amount of time that has elapsed since the outputting of the prompt;
comparing, by the processing device, the amount of time that has elapsed since the outputting of the prompt to a stored threshold amount of time;
determining, by the processing device, that the amount of time that has elapsed since the outputting of the prompt exceeds the stored threshold amount of time; and
transmitting, via a wireless communication device, an alert.

### Claim 15 (depends on 13)

The method for MR and AI-enhanced fire investigation of claim 13, wherein the identifying, by the processing device and by the execution of the AI safety criteria model, of the safety hazard in the environment, comprises:
identifying an attribute of the wearer;
comparing the attribute of the wearer to the 3-D wire mesh model descriptive of the environment; and
identifying, based on the comparing, a conflict between the attribute of the wearer and at least one portion of the 3-D wire mesh model.

### Claim 16 (depends on 15)

The method for MR and AI-enhanced fire investigation of claim 15, wherein the attribute of the wearer comprises at least one of a height and width of the wearer.

### Claim 17 (depends on 13)

The method for MR and AI-enhanced fire investigation of claim 13, wherein the safety hazard comprises an excessive thermal reading.

### Claim 18 (depends on 13)

The method for MR and AI-enhanced fire investigation of claim 13, wherein the safety hazard comprises at least one of an oxygen and a carbon dioxide reading that falls outside of a respective acceptable range.

### Claim 19 (depends on 13)

The method for MR and AI-enhanced fire investigation of claim 13, wherein the safety hazard comprises an exposed electrical wire.

### Claim 20 (depends on 13)

The method for MR and AI-enhanced fire investigation of claim 13, wherein the MR device comprises a battery in communication with at least one of the processing device, the ToF sensor, the head-mounted see-through display, the camera, and the one or more IMU devices.
