# US 2020/0394784 A1 · Artificial intelligence-based process and system for visual inspection of infrastructure

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394784 A1
- Document ID: US20200394784A1
- Status: Published application
- Kind: A1
- Application number: 16/904,111
- Filed: 2020-06-17
- Published: 2020-12-17
- Earliest claimed: 2019-06-17
- Assignees: RecognAIse Technologies Inc.
- Applicants: RecognAIse Technologies Inc.
- Inventors: Janos Csaba Toth; David Stefan Hauser; Attila Daniel Toth; Melinda Meszaros
- Web page: https://patentgrep.com/patent/US20200394784A1

## CPC

- G06T 7/0004: {Industrial image inspection}
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30108: Industrial image inspection
- G06T 2207/30184: Infrastructure
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An artificial intelligence (AI) based system for detecting defects in infrastructure uses an image recognizer and image data. A set of annotated training and validation data is generated to train and validate the image recognizer. The image data is annotated with classification data such as defect type and severity of the defect. Once trained and validated, the image recognizer can analyze inspection images to identify detects therein and generate an output report including the identification and classification of the defect, and remediation recommendations.

## Claims

### Claim 1

A method of automatically inspecting infrastructure, comprising:
receiving image data comprising images of infrastructure defects, annotated with classification data;
allocating a first portion of the image data as training image data;
allocating a second portion of the image data as validation image data;
training an artificial intelligence-based image recognizer using the training image data; and
validating the image recognizer using the validation image data.

### Claim 2 (depends on 1)

The method of claim 1 comprising:
inputting an inspection image into the image recognizer;
analyzing, by the image recognizer, the inspection image;
identifying, by the image recognizer, a defect in the inspection image;
locating in a database, by the image recognizer, a remediation recommendation associated with the defect; and
generating an output report based on the defect and the remediation recommendation.

### Claim 3 (depends on 2)

The method of claim 2 wherein the output report comprises identification and classification of the defect and one or more of: a potential cause of the defect; severity of the defect; and a recommendation on how to address the defect.

### Claim 4 (depends on 1)

The method of claim 1 wherein the classification data comprises defect type and/or severity.

### Claim 5 (depends on 1)

The method of claim 1 comprising providing a plurality of images of infrastructure, and filtering the plurality of images to generate a set of inspection images for input to the image recognizer.

### Claim 6 (depends on 5)

The method of claim 5 wherein filtering the plurality of images comprises one or more of: selecting images of high definition, selecting images that show a close-up image of the feature and/or defect, selecting images in which the feature is clearly visible without obstructions; and selecting images captured with good lighting conditions.

### Claim 7 (depends on 1)

The method of claim 1 wherein validating comprises:
calculating a recognition accuracy level of the image recognizer; and
comparing the recognition accuracy level with a threshold confidence level.

### Claim 8 (depends on 7)

The method of claim 7 comprising initially allocating approximately 70% of the image data as the training image data and allocating approximately 30% of the image data as the validation image data.

### Claim 9 (depends on 8)

The method of claim 8 comprising, if the recognition accuracy level is below a threshold confidence level, adjusting the allocation of training image data and validation image data to improve the recognition accuracy level.

### Claim 10 (depends on 4)

The method of claim 4 wherein the defect type is one or more of:
discoloration, fatigue and environmental.

### Claim 11

A system for automatically inspecting infrastructure, comprising:
a database management component configured to collect image data comprising images of infrastructure defects annotated with classification data, allocate a first portion of the image data as training image data, and allocate a second portion of the image data as validation image data; and
an artificial intelligence-based image recognizer configured to receive the image data from the database management component and apply deep learning techniques to develop processes to identify and classify infrastructure defects.

### Claim 12 (depends on 11)

The system of claim 11 wherein the image recognizer is configured to receive an inspection image, analyze the image, identify a defect in the inspection image, locate in a database a remediation recommendation associated with the defect, and generate an output report based on the defect and the remediation recommendation.

### Claim 13 (depends on 12)

The system of claim 12 wherein the output report comprises identification and classification of the defect and one or more of: a potential cause of the defect; severity of the defect; and a recommendation on how to address the defect.

### Claim 14 (depends on 11)

The system of claim 11 wherein the classification data comprises defect type and/or severity.

### Claim 15 (depends on 11)

The system of claim 11 comprising an image filtering component configured to receive a plurality of images of infrastructure, and filter the plurality of images to generate a set of inspection images for input to the image recognizer.

### Claim 16 (depends on 15)

The system of claim 15 wherein the image filtering component is configured to perform one or more of: selecting images of high definition, selecting images that show a close-up image of the feature and/or defect, selecting images in which the feature is clearly visible without obstructions; and selecting images captured with good lighting conditions.

### Claim 17 (depends on 12)

The system of claim 12 comprising a validation component configured to receive the output reports generated by the image recognizer using validation data, and compare them to the output reports generated by the image recognizer using the training data, calculate a recognition accuracy level of the image recognizer, and compare the recognition accuracy level with a threshold confidence level.

### Claim 18 (depends on 17)

The system of claim 17 wherein the database management component is configured to initially allocate approximately 70% of the image data as the training image data and allocate approximately 30% of the image data as the validation image data.

### Claim 19 (depends on 18)

The system of claim 18 wherein the validation component causes the database management component to adjust the allocation of training image data and validation image data to improve the recognition accuracy level if the recognition accuracy level is determined to be below a threshold confidence level.

### Claim 20 (depends on 14)

The system of claim 14 wherein the defect type is one or more of: discoloration, fatigue and environmental.