# US 12,118,703 B2 · Photo-realistic infrastructure inspection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,703 B2
- Document ID: US12118703B2
- Status: Granted
- Kind: B2
- Application number: 17/365,773
- Filed: 2021-07-01
- Published: 2022-01-27
- Granted: 2024-10-15
- Earliest claimed: 2020-07-02
- Assignees: RedZone Robotics, Inc.
- Applicants: RedZone Robotics, Inc.
- Inventors: Andrew Harmen; Gabe Detter; Mohamed Morsy; Andrew Lawniczak; Galin Konalchiev; Matthew Crowell; Jack Buffington; Mohsen Pashna; Richard Rigby; Jason Mizgorski; Foster J Salotti; Jjianan Lin; Valerie Walch
- Web page: https://patentgrep.com/patent/US12118703B2

## CPC

- G06T 7/0004: {Industrial image inspection}
- G06T 7/344: {involving models}
- G06T 7/55: from multiple images
- G06T 7/593: from stereo images
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- H04N 13/243: using three or more two-dimensional [2D] image sensors
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/30196: Human being; Person
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

One aspect provides a modular inspection robot for inspecting vertical shafts, chambers or tunnels. An embodiment provides related methods and products. One method includes: capturing, using a plurality of video cameras associated with an infrastructure inspection unit, two or more videos of infrastructure; accessing, using one or more processors, image metadata indicating a mesh of connected vertices based on the two or more videos; selecting, using the one or more processors, image data of frames of the two or more videos for inclusion in an output based on the mesh; and outputting, using the one or more processors, a photo-realistic image of the infrastructure comprising the image data selected. Other examples are described and claimed.

## Claims

### Claim 1

A method, comprising:
capturing, using one or more video cameras associated with an infrastructure inspection unit, two or more video frames of infrastructure;
accessing, using one or more processors, image metadata indicating a model of connected points based on the two or more video frames;
identifying, using the one or more processors, image data of the two or more video frames of the infrastructure;
selecting, using the one or more processors, image data of the two or more video frames for inclusion in an output based on the model;
wherein the selecting comprises selecting first image data of a single frame for inclusion in a photorealistic image; and
outputting, using the one or more processors, the photo-realistic image of the infrastructure comprising the image data selected.

### Claim 2 (depends on 1)

The method of claim 1, wherein the selecting is based on a point of view of a video camera of the one or more video cameras.

### Claim 3 (depends on 1)

The method of claim 1, wherein the photo-realistic image comprises the model.

### Claim 4 (depends on 3)

The method of claim 3, comprising providing, using the one or more processors, an interface element for culling data from the model.

### Claim 5 (depends on 3)

The method of claim 3, comprising providing, using the one or more processors, an interface element for indicating a portion of the model.

### Claim 6 (depends on 5)

The method of claim 5, comprising, responsive to receiving an indication via the interface element, providing quantitative measurement data for a feature of the infrastructure based on the model.

### Claim 7 (depends on 6)

The method of claim 6, comprising automatically scaling, based on the feature, quantitative measurement data for one or more other features of the infrastructure based on the indication.

### Claim 8 (depends on 1)

The method of claim 1, wherein the photo-realistic image is one or more of an image, a frame in a video, a virtual three-dimensional image, and the model comprising pixels of image data of the two or more video frames.

### Claim 9

A computer program product, comprising:
a non-transitory computer readable medium storing computer-executable code comprising:
code for accessing two or more video frames of infrastructure;
code for accessing image metadata indicating a model of connected points based on the two or more video frames;
code for identifying image data of the two or more video frames of the infrastructure;
code for selecting image data of the two or more video frames for inclusion in an output based on the model; and
wherein the selecting comprises selecting first image data of a single frame for inclusion in a photorealistic image;
code for outputting the photo-realistic image of the infrastructure comprising the image data selected.

### Claim 10 (depends on 9)

The computer program product of claim 9, wherein the selecting is based on a point of view of a video camera of one or more video cameras.

### Claim 11 (depends on 9)

The computer program product of claim 9, wherein the photo-realistic image comprises the model.

### Claim 12 (depends on 11)

The computer program product of claim 11, comprising code for providing an interface element for culling data from the model.

### Claim 13 (depends on 11)

The computer program product of claim 11, comprising code for providing an interface element for indicating a portion of the model.

### Claim 14 (depends on 13)

The computer program product of claim 13, comprising code for, responsive to receiving an indication via the interface element, providing quantitative measurement data for a feature of the infrastructure based on the model.

### Claim 15 (depends on 14)

The computer program product of claim 14, comprising code for automatically scaling, based on the feature, quantitative measurement data for one or more other features of the infrastructure based on the indication.

### Claim 16 (depends on 9)

The computer program product of claim 9, wherein the photo-realistic image is one or more of an image, a frame in a video, a virtual three-dimensional image, and the model comprising pixels of image data of the two or more video frames.

### Claim 17

A method, comprising:
obtaining, from one or more video cameras associated with an infrastructure inspection unit, two or more video frames of infrastructure;
accessing, using one or more processors, image metadata indicating overlap between the two or more video frames;
identifying, using the one or more processors, image data of the two or more video frames of the infrastructure;
selecting, using the one or more processors, image data of the two or more video frames for inclusion in an output based on the model;
wherein the selecting comprises selecting first image data of a single frame for inclusion in a photorealistic image; and
outputting, using the one or more processors, the photo-realistic image of the infrastructure comprising the image data selected.

### Claim 18 (depends on 3)

The method of claim 3, comprising providing, using the one or more processors, an interface element for interacting with the model.