# US 11,908,118 B2 · Visual model for image analysis of material characterization and analysis method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,118 B2
- Document ID: US11908118B2
- Status: Granted
- Kind: B2
- Application number: 17/387,955
- Filed: 2021-07-28
- Published: 2022-09-22
- Granted: 2024-02-20
- Earliest claimed: 2021-03-19
- Assignees: CITIC Dicastal Co., Ltd.
- Applicants: CITIC Dicastal Co., Ltd.
- Inventors: Zuo Xu; Yuancheng Cao; Wuxin Sha; Zhihua Zhu; Hanqi Wu; Fanpeng Cheng
- Web page: https://patentgrep.com/patent/US11908118B2

## CPC

- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06V 10/82: using neural networks
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06T 7/0004: {Industrial image inspection}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/277: involving stochastic approaches, e.g. using Kalman filters
- G06T 7/62: of area, perimeter, diameter or volume
- G06V 10/30: Noise filtering
- G06V 20/695: {Preprocessing, e.g. image segmentation}
- G06V 20/698: {Matching; Classification}
- G06T 2207/10056: Microscopic image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30241: Trajectory
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present disclosure provides a visual model for image analysis of material characterization and analysis method thereof. By collecting and labeling big data of microscopic images, the present disclosure establishes an image data set of material characterization; and uses this data set for high-throughput deep learning, establishes a neural network model and dynamic statistical model based on deep learning, to identify and locate atomic or lattice defects, and automatically mark the lattice spacing, obtain the classification and statistics of the true shape of the microscopic particles of the material, quantitatively analyze the tissue dynamics of the material.

## Claims

### Claim 1

A method for analyzing images of material characterization, wherein it comprises the following steps:
collect and mark samples of material characterization images to establish an image data set of material characterization, wherein the said image data set of material characterization comprises a first data set composed of first material characterization images marked with atomic species and crystal structure parameters, and a second data set composed of second material characterization images marked with edges and center points of microscopic particles and a third data set composed of third material characterization images marked with microstructure features;
establish an initial neural network model, and use the image data set of material characterization to train it to obtain a neural network model based on deep learning; and establish a dynamic statistical model;
input the characterization image of the material to be analyzed into the deep learning-based neural network model, and identify and analyze the output results, complete atom identification and interplanar spacing annotation, microscopic particle morphology statistics; and
input the third data set or output data of the deep learning-based neural network model into the dynamic statistical model to complete tracking of microstructure motion trajectory.

### Claim 2 (depends on 1)

The method for analyzing images of material characterization according to claim 1, wherein the said deep learning-based neural network model comprises a first neural network model trained on the said first data set and a second neural network model trained on the said second data set.

### Claim 3 (depends on 2)

The method for analyzing images of material characterization according to claim 2, wherein the training of the said first neural network model comprises the following steps:
S1. Input a pixel matrix of any one of the first material characterization images in the said first data set into the initial neural network model, and after the pixel values are sequentially calculated through the convolutional layers, the pooling layers, the fully connected layers, and the fully convolutional layers, an output pixel matrix is obtained;
S2. Compare the said output pixel matrix obtained in step S1 with the pixel matrix of the said first material characterization image used in step S1, and calculate a loss function value;
S3. Repeat steps S1 to S2 and input the pixel matrices of the said other first material characterization images in the said first data set into the said initial neural network model to obtain corresponding loss function value related to each of the said first material characterization image; then calculate the loss gradient according to each loss function value, and use a chain method to adjust network parameters in the said initial neural network model to obtain an adjusted neural network model;
S4. Replace the said initial neural network model with the said adjusted neural network model, and repeat steps S1 to S3 to adjust the said network parameters until the obtained loss function value is within a predetermined threshold range, thus finishing the training of the first neural network model.

### Claim 4 (depends on 2)

The method for analyzing images of material characterization according to claim 2, wherein the output result of the said first neural network model comprises a segmented atomic image, a lattice image, and the identified atomic species and crystal structure parameters, the said crystal structure parameters comprise interplanar spacing, cell parameters and crystal orientation; when recognizing and analyzing the output result of the said first neural network model, it also comprises judging the types of atomic defects and lattice defects.

### Claim 5 (depends on 3)

The method for analyzing images of material characterization according to claim 3, wherein the output result of the said first neural network model comprises a segmented atomic image, a lattice image, and the identified atomic species and crystal structure parameters, the said crystal structure parameters comprise interplanar spacing, cell parameters and crystal orientation; when recognizing and analyzing the output result of the said first neural network model, it also comprises judging the types of atomic defects and lattice defects.

### Claim 6 (depends on 2)

The method for analyzing images of material characterization according to claim 2, wherein the said second material characterization image in the said second data set is obtained according to the following method:
After performing noise reduction pretreatment on the material characterization image containing microscopic particles, the said microscopic particles in the image are separated from the background, and the edges and center points of each of the said microscopic particles are manually labeled;
When the said microscopic particles overlap or are adjacent to each other, multiple image transformation methods are used to separate and extract them.

### Claim 7 (depends on 4)

The method for analyzing images of material characterization according to claim 4, wherein the said second material characterization image in the said second data set is obtained according to the following method:
After performing noise reduction pretreatment on the material characterization image containing microscopic particles, the said microscopic particles in the image are separated from the background, and the edges and center points of each of the said microscopic particles are manually labeled;
When the said microscopic particles overlap or are adjacent to each other, multiple image transformation methods are used to separate and extract them.

### Claim 8 (depends on 6)

The method for analyzing images of material characterization according to claim 6, wherein the said second neural network model is used to identify and separate the edges and centers of the said microscopic particles, and output the said morphological characteristics of the microscopic particles; the said morphological characteristics comprise equivalent diameter, area, perimeter and curvature; when recognizing and analyzing the output result of the said second neural network model, it also comprises establishing an association database between the said morphological characteristics and the macroscopic properties of the material.

### Claim 9 (depends on 7)

The method for analyzing images of material characterization according to claim 7, wherein the said second neural network model is used to identify and separate the edges and centers of the said microscopic particles, and output the said morphological characteristics of the microscopic particles; the said morphological characteristics comprise equivalent diameter, area, perimeter and curvature; when recognizing and analyzing the output result of the said second neural network model, it also comprises establishing an association database between the said morphological characteristics and the macroscopic properties of the material.

### Claim 10 (depends on 2)

The method for analyzing images of material characterization according to claim 2, wherein the said microstructure features marked in the said third material characterization image in the said third data set comprise the contour and category of the microstructure; the said dynamic statistical model is used to deal with the said third data set or the output data of the said second neural network model, identify and track the position and destination of the microstructure.

### Claim 11 (depends on 1)

The method for analyzing images of material characterization according to claim 1, wherein the output result of the said dynamic statistical model comprises the motion trajectory of the microstructure; an unsupervised method is used to identify and analyze the output result of the said dynamic statistical model for identifying the material structure of different energy states, and the tissue dynamics of the material is quantitatively analyzed according to the said motion trajectory, so as to calculate and derive the macroscopic characteristics of the material.

### Claim 12

A visual model for image analysis of material characterization, wherein it comprises:
a first neural network model used to complete atom identification and interplanar spacing annotation, the first neural network model is trained on a first data set, the first data set is composed of first material characterization images marked with atomic species and crystal structure parameters,
a second neural network model used to count microscopic particle morphology, the second neural network model is trained on a second data set, the second data set is composed of second material characterization images marked with edges and center points of microscopic particles; and
a dynamic statistical model used to deal with the said third data set or the output data of the said second neural network model to track the motion trajectory of microstructures, the third data set composed of third material characterization images marked with microstructure features.