# US 12,223,016 B2 · System and method using machine learning for automated casing and/or tubing inspection log interpretation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,016 B2 - Document ID: US12223016B2 - Status: Granted - Kind: B2 - Application number: 17/659,215 - Filed: 2022-04-14 - Published: 2022-10-20 - Granted: 2025-02-11 - Earliest claimed: 2021-04-16 - Assignees: Saudi Arabian Oil Company - Applicants: Saudi Arabian Oil Company - Inventors: Chicheng Xu; Weichang Li; Olanrewaju A. Abudu; Shouxiang Mark Ma - Web page: https://patentgrep.com/patent/US12223016B2 ## CPC - G06F 18/2431: Multiple classes - G06F 16/55: Clustering; Classification - G06T 7/001: {using an image reference approach} - G06V 10/764: using classification, e.g. of video objects - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 10/778: Active pattern-learning, e.g. online learning of image or video features - G06V 10/82: using neural networks - G06T 2207/10132: Ultrasound image - G06T 2207/20036: Morphological image processing - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30108: Industrial image inspection - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computer-implemented method includes: accessing a first database holding information encoding a set of labels that specify a condition of at least one of: a surface pipe, or an underground enclosure that runs at a plurality of depth locations; accessing a second database holding a plurality of inspection logs that record measurement data of the surface pipe or underground enclosure; based on, at least in part, the labeling information and the plurality of inspection logs, training a deep learning model configured to classify, into the set of labels, the condition of the surface pipe or underground enclosure when presented with the inspection logs; applying the deep learning model to one or more newly received inspection logs containing measurement data of a new surface pipe or a new underground enclosure; and subsequently classifying, into the set of labels, the condition of the new surface pipe or the new underground enclosure. ## Claims ### Claim 1 A computer-implemented method comprising: accessing a first database holding information encoding a set of labels, wherein the set of labels specify a condition of at least one of: a surface pipe, or an underground enclosure, wherein the underground enclosure includes a casing or a tubing, either of which runs at a plurality of depth locations; accessing a second database holding a plurality of inspection logs, wherein the plurality of inspection logs record measurement data of the at least one of the surface pipe or the underground enclosure that runs at the plurality of depth locations; based on, at least in part, the set of labels and the plurality of inspection logs, training a deep learning model configured to classify, into the set of labels, the condition of the at least one of the surface pipe or the underground enclosure that runs at the plurality of depth locations, when presented with the inspection logs; applying the deep learning model to one or more newly received inspection logs containing measurement data of a new surface pipe or a new underground enclosure; and subsequently classifying, into the set of labels, the condition of the new surface pipe or the new underground enclosure. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the condition of the surface pipe or the underground enclosure characterizes an integrity state of the surface pipe or the underground enclosure. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the set of labels of the first database is generated based on, at least in part, input from one or more human experts when presented with at least portions of the inspection logs. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the plurality of inspection logs comprise logs from at least one of: a multi-finger cased-hole caliper tool, a flux-leakage tool, an electromagnetic (EM) phase shift tool, and an ultrasonic imaging tool. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the plurality of inspection logs include at least one of a low-frequency thickness image and at least one of a high-frequency discrimination image, and wherein the low-frequency thickness image and the high-frequency discrimination image are based on, at least in part, the EM phase shift tool. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the deep learning model includes a UNet classifier. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, wherein the UNet classifier comprises: a first stage configured to perform a pixel-level classification and classify each pixel or each patch of pixels into one of a set of integrity labels, and a second stage of using morphological patterns to discriminate defect patterns according to a respective size of each defect pattern. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the deep learning model includes a patch-based image classifier configured to operate on patches of pixels. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, wherein the patch-based image classifier incorporates a convolutional neural network (CNN) classifier. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, wherein the CNN classifier comprises: classifying patches of each inspection image into one of a set of integrity states; assigning the classified integrity state to a center pixel of the patch; and averaging over adjacent patches. ### Claim 11 A computer system comprising one or more computer processors configured to perform operations of: accessing a first database holding information encoding a set of labels, wherein the set of labels specify a condition of at least one of: a surface pipe, or an underground enclosure, wherein the at least one underground enclosure includes a tubing or a casing, either of which runs at a plurality of depth locations; accessing a second database holding a plurality of inspection logs, wherein the plurality of inspection logs record measurement data of the at least one of the surface pipe or the underground enclosure that runs at the plurality of depth locations; based on, at least in part, the set of labels and the plurality of inspection logs, training a deep learning model configured to classify, into the set of labels, the condition of the at least one of the surface pipe or the underground enclosure that runs at the plurality of depth locations, when presented with the inspection logs; applying the deep learning model to one or more newly received inspection logs containing measurement data of a new surface pipe or a new underground enclosure; and subsequently classifying, into the set of labels, the condition of the new surface pipe or the new underground enclosure. ### Claim 12 (depends on 11) The computer system of claim 11, wherein the condition of the surface pipe or the underground enclosure characterizes an integrity state of the surface pipe or the underground enclosure. ### Claim 13 (depends on 11) The computer system of claim 11, wherein the set of labels of the first database is generated based on, at least in part, input from one or more human experts when presented with at least portions of the inspection logs. ### Claim 14 (depends on 11) The computer system of claim 11, wherein the plurality of inspection logs comprise logs from at least one of: a multi-finger cased-hole caliper tool, a flux-leakage tool, an electromagnetic (EM) phase shift tool, and an ultrasonic imaging tool. ### Claim 15 (depends on 14) The computer system of claim 14, wherein the plurality of inspection logs include at least one of a low-frequency thickness image and at least one of a high-frequency discrimination image, and wherein the low-frequency thickness image and the high-frequency discrimination image are based on, at least in part, the EM phase shift tool. ### Claim 16 (depends on 11) The computer system of claim 11, wherein the deep learning model includes a UNet classifier. ### Claim 17 (depends on 16) The computer system of claim 16, wherein the UNet classifier comprises: a first stage configured to perform a pixel-level classification and classify each pixel or each patch of pixels into one of a set of integrity labels, and a second stage of using morphological patterns to discriminate defect patterns according to a respective size of each defect pattern. ### Claim 18 (depends on 11) The computer system of claim 11, wherein the deep learning model includes a patch-based image classifier configured to operate on patches of pixels. ### Claim 19 (depends on 18) The computer system of claim 18, wherein the patch-based image classifier incorporates a convolutional neural network (CNN) classifier. ### Claim 20 (depends on 19) The computer system of claim 19, wherein the CNN classifier comprises: classifying patches of each inspection image is into one of a set of integrity labels; assigning the classified integrity state to a center pixel of the patch; and averaging over adjacent patches.