# US 2024/0202901 A1 · Image processing of drone-captured image for asset management

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202901 A1
- Document ID: US20240202901A1
- Status: Published application
- Kind: A1
- Application number: 18/081,945
- Filed: 2022-12-15
- Published: 2024-06-20
- Earliest claimed: 2022-12-15
- Assignees: Airtonomy, Inc.
- Applicants: Airtonomy, Inc.
- Inventors: Travis James Desell
- Web page: https://patentgrep.com/patent/US20240202901A1

## CPC

- G06T 7/0004: {Industrial image inspection}
- G06T 7/001: {using an image reference approach}
- B64C 39/024: {of the remote controlled vehicle type, i.e. RPV}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/74: {involving reference images or patches}
- G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching}
- G06V 20/17: taken from planes or by drones
- G06V 20/176: {Urban or other man-made structures}
- B64U 2101/26: for manufacturing, inspections or repairs
- B64U 2101/30: for imaging, photography or videography
- B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/30184: Infrastructure
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B64C: AEROPLANES; HELICOPTERS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR

## Abstract

Methods, systems, and computer programs are presented for inspecting an asset using drone imagery. One method includes an operation for identifying an approximate location of an asset in an image, and for defining parameters of a mask associated with the asset in the image. The method further includes operations for performing a global optimization method to determine values for the parameters to obtain an optimized mask that corresponds to the asset in the image, and for extracting pixels of the image using the optimized mask to obtain asset pixels. The method further includes performing damage analysis for the asset based on the extracted pixels, and presenting results of the damage analysis on a display.

## Claims

### Claim 1

A computer-implemented method comprising:
identifying an approximate location of an asset in an image;
defining parameters of a mask associated with the asset in the image;
performing a global optimization method to determine values for the parameters to obtain an optimized mask that corresponds to the asset in the image;
extracting pixels of the image using the optimized mask to obtain asset pixels;
performing damage analysis for the asset based on the extracted pixels; and
presenting results of the damage analysis on a display.

### Claim 2 (depends on 1)

The method as recited in claim 1, wherein the parameters comprise:
horizontal offset and vertical offset of a top right corner in the image;
horizontal shift for four corner locations of the mask; and
vertical shift for the four corner locations of the mask.

### Claim 3 (depends on 2)

The method as recited in claim 2, wherein the asset is a string of solar panels, wherein the parameters further comprise a panel width adjustment, and a panel height adjustment.

### Claim 4 (depends on 3)

The method as recited in claim 3, the method further comprising:
extracting pixels for each of the panels in the string of solar panels, wherein performing damage analysis for the asset comprises performing damage analysis for each panel based on the extracted pixels for the panel.

### Claim 5 (depends on 2)

The method as recited in claim 2, wherein the asset is wind turbine, wherein the parameters further comprise a rotation angle of a spinner in the wind turbine.

### Claim 6 (depends on 1)

The method as recited in claim 1, wherein extracting pixels of the image further comprises:
performing a bitwise logical AND between corresponding pixels in the image and the mask.

### Claim 7 (depends on 1)

The method as recited in claim 1, wherein the global optimization method is a particle swarm optimization based on hill climbing.

### Claim 8 (depends on 1)

The method as recited in claim 1, further comprising:
before identifying the approximate location of the asset, capturing the image the asset with an autonomous drone.

### Claim 9 (depends on 8)

The method as recited in claim 8, wherein identifying the approximate location of the asset comprises:
determining the approximate location of the asset based on a location of the drone when taking the image.

### Claim 10 (depends on 1)

The method as recited in claim 1, wherein performing damage analysis comprises:
calculating an average pixel value of the pixels in the asset; and
determining pixels that deviate a predetermined amount from the average pixel value.

### Claim 11

A system comprising:
a memory comprising instructions; and
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising:
identifying an approximate location of an asset in an image;
defining parameters of a mask associated with the asset in the image;
performing a global optimization method to determine values for the parameters to obtain an optimized mask that corresponds to the asset in the image;
extracting pixels of the image using the optimized mask to obtain asset pixels;
performing damage analysis for the asset based on the extracted pixels; and
presenting results of the damage analysis on a display.

### Claim 12 (depends on 11)

The system as recited in claim 11, wherein the parameters comprise:
horizontal offset and vertical offset of a top right corner in the image;
horizontal shift for four corner locations of the mask; and
vertical shift for the four corner locations of the mask.

### Claim 13 (depends on 12)

The system as recited in claim 12, wherein the asset is a string of solar panels, wherein the parameters further comprise a panel width adjustment, and a panel height adjustment.

### Claim 14 (depends on 13)

The system as recited in claim 13, wherein the instructions further cause the one or more computer processors to perform operations comprising:
extracting pixels for each of the panels in the string of solar panels, wherein performing damage analysis for the asset comprises performing damage analysis for each panel based on the extracted pixels for the panel.

### Claim 15 (depends on 12)

The system as recited in claim 12, wherein the asset is wind turbine, wherein the parameters further comprise a rotation angle of a spinner in the wind turbine.

### Claim 16

A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
identifying an approximate location of an asset in an image;
defining parameters of a mask associated with the asset in the image;
performing a global optimization method to determine values for the parameters to obtain an optimized mask that corresponds to the asset in the image;
extracting pixels of the image using the optimized mask to obtain asset pixels;
performing damage analysis for the asset based on the extracted pixels; and
presenting results of the damage analysis on a display.

### Claim 17 (depends on 16)

The tangible machine-readable storage medium as recited in claim 16, wherein the parameters comprise:
horizontal offset and vertical offset of a top right corner in the image;
horizontal shift for four corner locations of the mask; and
vertical shift for the four corner locations of the mask.

### Claim 18 (depends on 17)

The tangible machine-readable storage medium as recited in claim 17, wherein the asset is a string of solar panels, wherein the parameters further comprise a panel width adjustment, and a panel height adjustment.

### Claim 19 (depends on 18)

The tangible machine-readable storage medium as recited in claim 18, wherein the machine further performs operations comprising:
extracting pixels for each of the panels in the string of solar panels, wherein performing damage analysis for the asset comprises performing damage analysis for each panel based on the extracted pixels for the panel.

### Claim 20 (depends on 17)

The tangible machine-readable storage medium as recited in claim 17, wherein the asset is wind turbine, wherein the parameters further comprise a rotation angle of a spinner in the wind turbine.
