# US 12,118,779 B1 · System and method for assessing structural damage in occluded aerial images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,779 B1
- Document ID: US12118779B1
- Status: Granted
- Kind: B1
- Application number: 17/936,551
- Filed: 2022-09-29
- Granted: 2024-10-15
- Earliest claimed: 2021-09-30
- Assignees: United Services Automobile Association (USAA)
- Applicants: Uipco, LLC
- Inventors: Yangqiu Hu; David Lamar Rogers; Ting Lu; Jess W. Gingrich
- Web page: https://patentgrep.com/patent/US12118779B1

## CPC

- G06V 20/176: {Urban or other man-made structures}
- G06Q 10/10: Office automation; Time management
- G06Q 40/08: Insurance
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 20/13: Satellite images
- G06V 20/17: taken from planes or by drones
- G16H 30/40: for processing medical images, e.g. editing
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A system and method for processing occluded aerial images of structures to assess structural damage is disclosed. The system employs a generative model that is trained on data created by adding occlusive features to clear aerial images. Both the clear images and their simulated occluded counterparts (image pairs) are inputted into the generative model as training data, and the model learns to recognize and discriminate patterns between the pairs. This process is fully automated and does not require manual labeling. Following training, the model is capable of receiving real (non-simulated) occluded images and generating a clarified image that can reveal structural details previously not apparent to automatically determine a damage classification for the property.

## Claims

### Claim 1

A method of identifying property damage in occluded aerial imagery, the method comprising:
receiving, at a damage classification system, first data including a first occluded image of a first property comprising at least a first structure, wherein the first structure is veiled in the first occluded image;
receiving, at the damage classification system, a training dataset, the training dataset comprising pairs of images, each pair including a first image and a second image, the first image providing a high visibility picture of a property, the second image providing a low visibility picture of the same property in the same position as in the first image;
wherein the second image is created employing an occlusion simulation engine that uses fractal expansion techniques to add occlusive features to the first image;
reducing a dynamic range of pixel intensities of the second image from a full intensity scale to a randomly compressed intensity scale;
feeding, at the damage classification system, the first occluded image to a generative model to generate a first reconstructed image in which the first structure is unveiled;
training, at the damage classification system, the generative model using the training dataset to produce high visibility pictures from occluded imagery;
extracting, at the damage classification system, a first feature from the first recon structed image;
estimating, by the damage classification system, damage to the first property based on at least the first feature;
classifying, by the damage classification system, the first property depicted in the first reconstructed image under a damage category based on the estimated damage; and
generating, at the damage classification system, a damage score for the first property based on the first reconstructed image; and
selecting, at the damage classification system, the damage category based on the damage score;
causing to be presented, at a computing device, the damage category, wherein at least a portion of the method is performed by a processing unit.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first occluded image is received from one of a drone, aircraft, and satellite further.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first structure is veiled in the first occluded image by one or more of the group comprising haze, smoke, rain, fog, and low lighting.

### Claim 4 (depends on 1)

The method of claim 1, further comprising employing, at the damage classification system, a machine learning model to detect damage to the first structure in the first reconstructed image.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
transmitting the first reconstructed image to one or a repair facility and an insurance representative.

### Claim 6 (depends on 5)

The method of claim 5, wherein the damage score is greater than a pre-set damage threshold, and the method further comprises:
generating, at the damage classification system, an output indicating the damage score is above the threshold;
transmitting, from the damage classification system, the output to a claims processing module; and
causing, at the claims processing module and in response to the output, a payout signal to be generated that is configured to automatically release funds to a beneficiary of an insurance policy for the first property.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
generating, at the damage classification system, an output describing the damage to the first property;
transmitting, from the damage classification system, the output to a claims processing module; and
causing, at the claims processing module and in response to the output, a preliminary claim to be generated that is configured to expedite a payout to a beneficiary of an insurance policy for the first property.

### Claim 8

A method of clarifying occluded images to determine property damage, the method comprising:
receiving, at a damage classification system, a training dataset, the training dataset comprising pairs of images, each pair including a first image and a second image, the first image providing a high visibility picture of a property, the second image providing a low visibility picture of the same property in the same position as in the first image;
wherein the second image is created employing an occlusion simulation engine that uses fractal expansion techniques to add occlusive features to the first image;
reducing a dynamic range of pixel intensities of the second image from a full intensity scale to a randomly compressed intensity scale;
training, at the damage classification system, a generative model using the training dataset to produce high visibility pictures from occluded imagery;
receiving, at the damage classification system, first data including a first occluded image of a first property comprising at least a first structure, wherein the first structure is veiled in the first occluded image;
feeding, at the damage classification system, the first occluded image to the generative model to generate a first reconstructed image in which the first structure is unveiled;
generating, at the damage classification system, a damage score for the first property based on the first reconstructed image; and
selecting, at the damage classification system, the damage category based on the damage score; and
causing to be presented, at a computing device, the first recon structed image.

### Claim 9 (depends on 8)

The method of claim 8, wherein the first data further includes a location of the first property, and the method further comprises causing to be presented, at the computing device, the first occluded image and information about an account associated with the location.

### Claim 10 (depends on 8)

The method of claim 8, further comprising:
extracting, at the damage classification system, a first feature from the first recon structed image; and
estimating, by the damage classification system, damage to the first property based on at least the first feature.

### Claim 11 (depends on 10)

The method of claim 10, further comprising classifying, by the damage classification system, the first property depicted in the first reconstructed image under a damage category based on the estimated damage.

### Claim 12 (depends on 11)

The method of claim 11, further comprising causing to be presented, at the computing device, the damage category.

### Claim 13 (depends on 10)

The method of claim 10, further comprising:
generating, at the damage classification system, an output describing the estimated damage to the first property;
transmitting, from the damage classification system, the output to a claims processing module; and
causing, at the claims processing module and in response to the output, a preliminary claim to be generated that is configured to expedite a payout to an owner of the first property.

### Claim 14

A system for identifying property damage in occluded aerial imagery, the system comprising a processor and machine-readable media including instructions which, when executed by the processor, cause the processor to:
receive, at a damage classification system, first data including a first occluded image of a first property comprising at least a first structure, wherein the first structure is veiled in the first occluded image;
receive, at the damage classification system, a training dataset, the training dataset comprising pairs of images, each pair including a first image and a second image, the first image providing a high visibility picture of a property, the second image providing a low visibility picture of the same property in the same position as in the first image;
wherein the second image is created employing an occlusion simulation engine that uses fractal expansion techniques to add occlusive features to the first image;
reduce a dynamic range of pixel intensities of the second image from a full intensity scale to a randomly compressed intensity scale;
feed, at the damage classification system, the first occluded image to a generative model to generate a first reconstructed image in which the first structure is unveiled;
train, at the damage classification system, the generative model using the training dataset to produce high visibility pictures from occluded imagery;
extract, at the damage classification system, a first feature from the first recon structed image;
estimate, by the damage classification system, damage to the first property based on at least the first feature;
classify, by the damage classification system, the first property depicted in the first reconstructed image under a damage category based on the estimated damage;
generate, at the damage classification system, a damage score for the first property based on the first reconstructed image; and
select, at the damage classification system, the damage category based on the damage score; and
cause to be presented, at a computing device, the damage category.

### Claim 15 (depends on 14)

The system of claim 14, wherein the instructions further cause the processor to:
transmit the first reconstructed image to one or a repair facility and an insurance representative.

### Claim 16 (depends on 14)

The system of claim 14, wherein the first structure is veiled in the first occluded image by one or more of the group comprising haze, smoke, rain, fog, and low lighting.

### Claim 17 (depends on 14)

The system of claim 14, wherein the instructions further cause the processor to employ, at the damage classification system, a machine learning model to detect damage to the first structure in the first reconstructed image.

### Claim 18 (depends on 14)

The system of claim 14, wherein the first occluded image is received from one of a drone, aircraft, and satellite.

### Claim 19 (depends on 18)

The system of claim 18, wherein the instructions further cause the processor to:
generate, at the damage classification system, an output indicating the damage score is above the threshold;
transmit, from the damage classification system, the output to a claims processing module; and
cause, at the claims processing module and in response to the output, a payout signal to be generated that is configured to automatically release funds to a beneficiary of an insurance policy for the first property.

### Claim 20 (depends on 14)

The system of claim 14, wherein the instructions further cause the processor to:
generate, at the damage classification system, an output describing the damage to the first property;
transmit, from the damage classification system, the output to a claims processing module; and
cause, at the claims processing module and in response to the output, a preliminary claim to be generated that is configured to expedite a payout to a beneficiary of an insurance policy for the first property.