# US 10,768,094 B2 · Thermography image processing with neural networks to identify corrosion under insulation (CUI)

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,768,094 B2
- Document ID: US10768094B2
- Status: Granted
- Kind: B2
- Application number: 16/711,099
- Filed: 2019-12-11
- Published: 2020-04-16
- Granted: 2020-09-08
- Earliest claimed: 2017-09-22
- Assignees: Saudi Arabian Oil Company
- Applicants: Saudi Arabian Oil Company
- Inventors: Ayman Amer; Ali Al Shehri; Brian Parrott; Mohammed Sarraj
- Web page: https://patentgrep.com/patent/US10768094B2

## CPC

- G01N 17/04: Corrosion probes
- G01K 1/045: {temperature indication combined with the indication of another variable (indicating of human comfort G01W1/17)}
- G01N 25/72: Investigating presence of flaws
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G01N 27/90: using eddy currents
- G06F 16/285: {Clustering or classification}
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G01K: MEASURING TEMPERATURE; MEASURING QUANTITY OF HEAT; THERMALLY-SENSITIVE ELEMENTS NOT OTHERWISE PROVIDED FOR (radiation pyrometry G01J5/00)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method for identifying corrosion under insulation (CUI) in a structure comprises receiving thermographs from the structure using an infrared camera, applying filters to the thermograph using a first machine learning system, initially determining a CUI classification based on output from the filters, and validating the initial CUI classification by an inspection of the structure. The first machine learning system is trained using results of the validation. Outputs of the first machine learning system and additional structural and environmental data are fed into a second machine learning system that incorporates information from earlier states into current states. The second machine learning system is trained to identify CUI according to changes in the outputs of the first machine learning system and the additional data over time until a second threshold for CUI classification accuracy is reached. CUI is thereafter identified using the first and second machine learning systems in coordination.

## Claims

### Claim 1

A computer-implemented method for identifying corrosion under insulation (CUI) in a structure using filters trained by a first machine learning system to a first threshold for CUI classification accuracy, comprising:
receiving a thermograph captured from the structure using an infrared radiation sensor and additional data related to the structure and environmental conditions;
inputting outputs of the first machine learning system and additional data related to the structure and environment conditions into a second machine learning system that incorporates information from earlier states into current states;
training the second machine learning system to identify CUI according to changes in the outputs of the first machine learning system and the additional data over time until a second threshold for CUI classification accuracy is reached; and
after the second threshold is reached, identifying CUI in the structure based on received thermograph and additional data using the first and second machine learning systems in coordination.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the first machine learning system includes a convolutional neural network.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the convolutional neural network includes a plurality of hierarchical layers, each hierarchical layer including a convolutional stage, a non-linear function stage and a pooling stage.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, wherein the second machine learning system includes a recurrent neural network.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein the additional data includes ambient temperature, physical characteristics of the structure and weather conditions measured over time.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein the first and second machine learning systems are trained to recognize false positive findings relative to reflection of infrared radiation from objects external from the structure.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein identification of CUI includes identifying vulnerable areas of the structure under the insulation that confine moisture with a high likelihood.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising:
validating an initial CUI classification by an inspection of the structure, wherein validation is performed using at least two of pulsed eddy current evaluation, visual inspection, insulation removal and ultrasonic testing of wall thinning.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, further comprising preprocessing the thermograph data and the additional data to encode categorical variables and normalize continuous variables.

### Claim 10

A system for identifying corrosion under insulation (CUI) in a structure using filters trained by a first machine learning system to a first threshold for CUI classification accuracy, comprising:
a computer system including a processor, memory and a communication module, the processor being configured to execute a program that performs steps of:
receiving a thermograph captured from the structure using an infrared radiation sensor and additional data related to the structure and environmental conditions;
inputting outputs of the first machine learning system and additional data related to the structure and environment conditions into a second machine learning system that incorporates information from earlier states into current states;
training the second machine learning system to identify CUI according to changes in the outputs of the first machine learning system and the additional data over time until a second threshold for CUI classification accuracy is reached; and
after the first and second thresholds are reached, identifying CUI in the structure based on current thermograph and additional data using the first and second machine learning systems in coordination.

### Claim 11 (depends on 10)

The system of claim 10, wherein the first machine learning system includes a convolutional neural network.

### Claim 12 (depends on 11)

The system of claim 11, wherein the convolutional neural network includes a plurality of hierarchical layers, each hierarchical layer including a convolutional stage, a non-linear function stage and a pooling stage.

### Claim 13 (depends on 10)

The system of claim 10, wherein the second machine learning system includes a recurrent neural network.

### Claim 14 (depends on 10)

The system of claim 10, wherein the additional data includes ambient temperature, physical characteristics of the structure and weather conditions measured over time.

### Claim 15 (depends on 10)

The system of claim 10, wherein the first and second machine learning systems are trained to recognize false positive findings relative to reflection of infrared radiation from objects external from the structure.

### Claim 16 (depends on 10)

The system of claim 10, wherein identification of CUI includes identifying vulnerable areas of the structure under the insulation that confine moisture with a high likelihood.

### Claim 17 (depends on 10)

The system of claim 10, wherein the processor is configured to execute the program that performs a further step of:
validating an initial CUI classification by an inspection of the structure, wherein validation is performed using at least two of pulsed eddy current evaluation, visual inspection, insulation removal and ultrasonic testing of wall thinning.

### Claim 18 (depends on 10)

The system of claim 10, wherein the computer system preprocesses the thermograph data and the additional data to encode categorical variables and normalize continuous variables.