# US 2018/0321692 A1 · Three-dimensional robotic inspection system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0321692 A1
- Document ID: US20180321692A1
- Status: Published application
- Kind: A1
- Application number: 15/587,950
- Filed: 2017-05-05
- Published: 2018-11-08
- Earliest claimed: 2017-05-05
- Assignees: General Electric Company
- Applicants: General Electric Company
- Inventors: Mauricio CASTILLO-EFFEN; Ching-Ling HUANG; Raju VENKATARAMANA; Roberto SILVA FILHO; Alex TEPPER; Steven GRAY; Yakov POLISHCHUK; Viktor HOLOVASHCHENKO; Charles THEURER; Yang ZHAO; Ghulam Ali BALOCH; Douglas FORMAN; Shiraj SEN; Huan TAN; Arpit JAIN
- Web page: https://patentgrep.com/patent/US20180321692A1

## CPC

- G06T 19/003: {Navigation within 3D models or images}
- G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target}
- G05D 1/101: {specially adapted for aircraft}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- B64U 2101/32: for cartography or topography
- Y10S 901/44: inspection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

Provided are systems and methods for monitoring an asset via an autonomous model-driven inspection. In an example, the method may include storing an inspection plan including a virtually created three-dimensional (3D) model of a travel path with respect to a virtual asset that is created in virtual space, converting the virtually created 3D model of the travel path about the virtual asset into a physical travel path about a physical asset corresponding to the virtual asset, autonomously controlling vertical and lateral movement of the unmanned robot in three dimensions with respect to the physical asset based on the physical travel path and capturing data at one or more regions of interest, and capturing data at one or more regions of interest, and storing information concerning the captured data about the asset.

## Claims

### Claim 1

A robotic inspection system, comprising:
a storage configured to store a three-dimensional (3D) model of a travel path with respect to a virtual asset that is created in virtual space; and
a processor configured to convert the virtually created 3D model of the travel path about the virtual asset into a physical travel path about a physical asset corresponding to the virtual asset, autonomously control vertical and lateral movement of the unmanned robot in three dimensions with respect to the physical asset based on the physical travel path, and capture data at one or more regions of interest,
wherein the storage is further configured to store information concerning the captured data about the asset.

### Claim 2 (depends on 1)

The robotic inspection system of claim 1, wherein the virtually created 3D model of the travel path is dynamically generated based on a plurality of user inputs with respect to a virtual replica of the physical asset displayed via a user interface.

### Claim 3 (depends on 1)

The robotic inspection system of claim 1, wherein the processor is configured to convert the virtually created 3D model of the travel path by aligning X, Y, and Z coordinates of the virtually created 3D model of the travel path in the virtual space into latitude, longitude, and height components of the physical travel path in physical space.

### Claim 4 (depends on 1)

The robotic inspection system of claim 1, wherein the processor is further configured to autonomously control a distance between the unmanned robot and a surface of the physical asset at a region of interest based on distance information included in the inspection plan.

### Claim 5 (depends on 1)

The robotic inspection system of claim 1, wherein the processor is further configured to autonomously control a field of view of an imaging capturing device of the unmanned robot with respect to a surface of the asset at a region of interest based on orientation information included in the inspection plan.

### Claim 6 (depends on 1)

The robotic inspection system of claim 1, wherein the processor is further configured to autonomously rotate the unmanned robot a predefined angle with respect to a fixed point on a surface of the asset at a region of interest based on rotation information included in the inspection plan.

### Claim 7 (depends on 1)

The robotic inspection system of claim 1, wherein the virtually created 3D model of the travel path comprises a dynamically selected virtual start point comprising a first vertical location and a first lateral location, and a dynamically selected virtual end point comprising a second vertical location and a second lateral location which differ from the first vertical location and the first lateral location, respectively.

### Claim 8 (depends on 7)

The robotic inspection system of claim 7, wherein the processor is further configured to autonomously move the unmanned robot from a physical start point of the physical travel path, around the asset one or more times, and to the physical end point of the physical travel path, based on the dynamically selected virtual start and end points of the virtually created 3D model of the travel path.

### Claim 9 (depends on 7)

The robotic inspection system of claim 7, wherein the processor is further configured to autonomously move the unmanned robot from a docking station to the physical starting point of the travel path based on the virtually created 3D model of the travel path.

### Claim 10

A method of an unmanned robot for monitoring an asset via an autonomous model-driven inspection, the method comprising:
storing a three-dimensional (3D) model of a travel path with respect to a virtual asset that is created in virtual space;
converting the virtually created 3D model of the travel path about the virtual asset into a physical travel path about a physical asset corresponding to the virtual asset;
autonomously controlling vertical and lateral movement of the unmanned robot in three dimensions with respect to the physical asset based on the physical travel path and capturing data at one or more regions of interest; and
storing information concerning the captured data about the physical asset.

### Claim 11 (depends on 10)

The method of claim 10, wherein the virtually created 3D model of the travel path is dynamically generated based on a plurality of user inputs with respect to a virtual replica of the physical asset displayed via a user interface.

### Claim 12 (depends on 10)

The method of claim 10, wherein the converting of the virtually created 3D model of the travel path comprises aligning X, Y, and Z coordinates of the virtually created 3D model of the travel path in the virtual space into latitude, longitude, and height components of the physical travel path in physical space.

### Claim 13 (depends on 10)

The method of claim 10, wherein the method further comprises autonomously controlling a distance between the unmanned robot and a surface of the physical asset at a region of interest based on distance information included in the inspection plan.

### Claim 14 (depends on 10)

The method of claim 10, wherein the method further comprises autonomously controlling a field of view of an imaging capturing device of the unmanned robot with respect to a surface of the physical asset at a region of interest based on orientation information included in the inspection plan.

### Claim 15 (depends on 10)

The method of claim 10, wherein the method further comprises autonomously rotating the unmanned robot a predefined angle with respect to a fixed point on a surface of the physical asset at a region of interest based on rotation information included in the inspection plan.

### Claim 16 (depends on 10)

The method of claim 10, wherein the virtually created 3D model of the travel path comprises a dynamically selected virtual start point comprising a first vertical location and a first lateral location, and a dynamically selected virtual end point comprising a second vertical location and a second lateral location which differ from the first vertical location and the first lateral location, respectively.

### Claim 17 (depends on 16)

The method of claim 16, wherein the method further comprises autonomously moving the unmanned robot from a physical start point of the physical travel path, around the asset one or more times, and to the physical end point of the physical travel path, based on the dynamically selected virtual start and end points of the virtually created 3D model of the travel path.

### Claim 18 (depends on 16)

The method of claim 16, wherein the method further comprises autonomously moving the unmanned robot from a docking station to the physical starting point of the travel path based on the virtually created 3D model of the travel path.

### Claim 19

A non-transitory computer readable medium having stored therein instructions that when executed cause a computer to perform a method of an unmanned robot for monitoring an asset via an autonomous model-driven inspection, the method comprising:
storing a three-dimensional (3D) model of a travel path with respect to a virtual asset that is created in virtual space;
converting the virtually created 3D model of the travel path about the virtual asset into a physical travel path about a physical asset corresponding to the virtual asset;
autonomously controlling vertical and lateral movement of the unmanned robot in three dimensions with respect to the physical asset based on the physical travel path and capturing data at one or more regions of interest; and
storing information concerning the captured data about the physical asset.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, wherein the virtually created 3D model of the travel path is dynamically generated based on a plurality of user inputs with respect to a virtual replica of the physical asset displayed via a user interface.
