# US 12,213,828 B2 · Image data inspection method and apparatus, computer device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,213,828 B2
- Document ID: US12213828B2
- Status: Granted
- Kind: B2
- Application number: 17/721,806
- Filed: 2022-04-15
- Published: 2022-07-28
- Granted: 2025-02-04
- Earliest claimed: 2020-04-30
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Shilei Cao; Hualuo Liu; Yefeng Zheng
- Web page: https://patentgrep.com/patent/US12213828B2

## CPC

- A61B 6/5241: {combining overlapping images of the same imaging modality, e.g. by stitching}
- A61B 6/032: {Transmission computed tomography [CT]}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/11: Region-based segmentation
- G06T 7/174: involving the use of two or more images
- G06V 10/16: {using multiple overlapping images; Image stitching}
- G06V 10/24: Aligning, centring, orientation detection or correction of the image
- G06V 10/40: Extraction of image or video features
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- G06T 2207/10072: Tomographic images
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30004: Biomedical image processing
- G06T 2207/30061: Lung
- G06V 2201/031: of internal organs
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

This application relates to an image data inspection method and apparatus in the field of artificial intelligence (AI) technologies. The method includes obtaining an image to be inspected, the image to be inspected comprising a sequence of slice images; determining a corresponding group of slice images for each target image in the sequence of slice images; extracting a corresponding slice feature map for each slice image in the group of slice images; aligning the slice feature maps extracted corresponding to the group of slice images; aggregating context information of each slice image in the group of slice images by using an aligned feature map; and performing target region inspection on an aggregated feature map, to obtain an inspection result corresponding to the target image, and combining the inspection result corresponding to each target image, to generate an inspection result corresponding to the image to be inspected.

## Claims

### Claim 1

An image data inspection method, performed by a computer, the method comprising:
obtaining an image to be inspected, the image to be inspected comprising a sequence of slice images;
determining a corresponding group of slice images for each target image in the sequence of slice images, the group of slice images comprising the target image and an adjacent image having a context relationship with the target image in the sequence of slice images;
extracting a corresponding slice feature map for each slice image in the group of slice images;
aligning the slice feature maps extracted corresponding to the group of slice images;
aggregating context information of each slice image in the group of slice images by using an aligned feature map; and
performing target region inspection on an aggregated feature map, to obtain an inspection result corresponding to the target image, and combining the inspection result corresponding to each target image, to generate an inspection result corresponding to the image to be inspected.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the aligning the slice feature maps extracted corresponding to the group of slice images comprises:
stitching the slice feature maps extracted corresponding to the group of slice images, and aligning the slice feature maps by using a stitched feature map, to generate the aligned feature map.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the stitching the slice feature maps extracted corresponding to the group of slice images comprises:
obtaining a slice quantity corresponding to the group of slice images;
stacking the slice feature maps extracted corresponding to the group of slice images, a stacked feature map comprising a slice quantity dimension and a channel dimension; and
exchanging a feature of the slice quantity dimension with a feature of the channel dimension, to generate the stitched feature map.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the stitched feature map further comprises a slice size dimension; and aligning the slice feature maps by using a stitched feature map comprises:
performing a three-dimensional convolution operation on the stitched feature map;
normalizing the slice quantity dimension and the slice size dimension of the feature map after the convolution operation; and
outputting the aligned feature map by performing nonlinear activation on the normalized feature map.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the aggregating context information of each slice image in the group of slice images by using an aligned feature map comprises:
reshaping the aligned feature map according to a slice quantity dimension, to generate a reshaped feature map;
weighting the aligned feature map by using the reshaped feature map; and
performing dimensionality reduction on the weighted feature map, to generate an aggregated feature map corresponding to the group of slice images.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the weighting the aligned feature map by using the reshaped feature map, and performing dimensionality reduction on the weighted feature map comprises:
transforming the reshaped feature map, and weighting the slice quantity dimension of the aligned feature map by using the transformed feature map; and
obtaining a dimensionality reduction convolution, and performing dimensionality reduction on the weighted feature map by using the dimensionality reduction convolution.

### Claim 7 (depends on 6)

The method according to claim 6, wherein a convolution kernel of the dimensionality reduction convolution is generated according to a slice quantity corresponding to the group of slice images.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the performing target region inspection on an aggregated feature map comprises:
identifying a category corresponding to a pixel falling within a target detection box in the aggregated feature map;
performing regression to obtain a bounding box of the pixel; and
predicting a relative distance between the pixel and a center point of the target detection box, and filtering out the bounding box of the pixel when the relative distance exceeds a preset distance.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising: training an image inspection model, the image inspection model comprising a backbone network, a three-dimensional alignment network, a three-dimensional aggregation network, and a target detection network, and the training comprising:
determining a corresponding group of slice sample images for each target sample image in a sequence of slice sample images of an image sample; the group of slice sample images comprising the target sample image and an adjacent sample image having a context relationship with the target sample image in the sequence of slice sample images;
extracting a corresponding slice sample feature map for each slice sample image in the group of slice sample images through the backbone network;
aligning the slice sample feature maps extracted corresponding to the group of slice sample images through the three-dimensional alignment network;
aggregating context information of each slice sample image in the group of slice sample images through the three-dimensional aggregation network by using an aligned sample feature map; and
performing target region inspection on an aggregated sample feature map through the target detection network, to obtain an inspection result corresponding to the target sample image, and combining the inspection result corresponding to each target sample image, to generate an inspection result corresponding to the image sample.

### Claim 10 (depends on 9)

The method according to claim 9, wherein aligning the slice sample feature maps extracted corresponding to the group of slice sample images through the three-dimensional alignment network comprises:
stitching the slice sample images extracted corresponding to the group of slice sample images through the three-dimensional alignment network, and aligning the slice feature maps by using a stitched sample feature map, to generate the aligned sample feature map.

### Claim 11 (depends on 9)

The method according to claim 9, wherein the aggregating context information of each slice sample image in the group of slice sample images through the three-dimensional aggregation network by using an aligned sample feature map comprises:
reshaping the aligned sample feature map according to a slice quantity dimension through the three-dimensional aggregation network, to generate a reshaped sample feature map;
weighting the aligned sample feature map by using the reshaped sample feature map; and
performing dimensionality reduction on the weighted sample feature map, to generate an aggregated sample feature map corresponding to the group of slice sample images.

### Claim 12 (depends on 9)

The method according to claim 9, wherein the target detection network comprises a classification branch, a regression branch, and a center point prediction branch; and the performing target region inspection on an aggregated sample feature map through the target detection network comprises:
identifying a category corresponding to a sample pixel falling within a target detection box in the aggregated sample feature map through the classification branch;
performing regression through the regression branch to obtain a bounding box of the sample pixel; and
predicting a relative distance between the sample pixel and a center point of the target detection box through the center point prediction branch, and filtering out the bounding box of the sample pixel when the relative distance exceeds a preset distance.

### Claim 13 (depends on 12)

The method according to claim 12, wherein the method further comprises:
obtaining a slice thickness corresponding to the slice sample image, and determining a slice adjustment quantity according to the slice thickness; and
performing a non-maximum suppression operation by using the slice adjustment quantity of slice sample images, to remove redundant detection boxes.

### Claim 14

One or more non-transitory storage media storing computer-readable instructions, the computer-readable instructions, when executed by one or more processors, causing the one or more processors to perform:
obtaining an image to be inspected, the image to be inspected comprising a sequence of slice images;
determining a corresponding group of slice images for each target image in the sequence of slice images, the group of slice images comprising the target image and an adjacent image having a context relationship with the target image in the sequence of slice images;
extracting a corresponding slice feature map for each slice image in the group of slice images;
aligning the slice feature maps extracted corresponding to the group of slice images;
aggregating context information of each slice image in the group of slice images by using an aligned feature map; and
performing target region inspection on an aggregated feature map, to obtain an inspection result corresponding to the target image, and combining the inspection result corresponding to each target image, to generate an inspection result corresponding to the image to be inspected.

### Claim 15 (depends on 14)

The storage media according to claim 14, wherein the aligning the slice feature maps extracted corresponding to the group of slice images comprises:
stitching the slice feature maps extracted corresponding to the group of slice images, and aligning the slice feature maps by using a stitched feature map, to generate the aligned feature map.

### Claim 16 (depends on 15)

The storage media according to claim 15, wherein the stitching the slice feature maps extracted corresponding to the group of slice images comprises:
obtaining a slice quantity corresponding to the group of slice images;
stacking the slice feature maps extracted corresponding to the group of slice images, a stacked feature map comprising a slice quantity dimension and a channel dimension; and
exchanging a feature of the slice quantity dimension with a feature of the channel dimension, to generate the stitched feature map.

### Claim 17 (depends on 16)

The storage media according to claim 16, wherein the stitched feature map further comprises a slice size dimension; and aligning the slice feature maps by using a stitched feature map comprises:
performing a three-dimensional convolution operation on the stitched feature map;
normalizing the slice quantity dimension and the slice size dimension of the feature map after the convolution operation; and
outputting the aligned feature map by performing nonlinear activation on the normalized feature map.

### Claim 18 (depends on 14)

The storage media according to claim 14, wherein the aggregating context information of each slice image in the group of slice images by using an aligned feature map comprises:
reshaping the aligned feature map according to a slice quantity dimension, to generate a reshaped feature map;
weighting the aligned feature map by using the reshaped feature map; and
performing dimensionality reduction on the weighted feature map, to generate an aggregated feature map corresponding to the group of slice images.

### Claim 19 (depends on 18)

The storage media according to claim 18, wherein the weighting the aligned feature map by using the reshaped feature map, and performing dimensionality reduction on the weighted feature map comprises:
transforming the reshaped feature map, and weighting the slice quantity dimension of the aligned feature map by using the transformed feature map; and
obtaining a dimensionality reduction convolution, and performing dimensionality reduction on the weighted feature map by using the dimensionality reduction convolution.

### Claim 20

An image data inspection apparatus, comprising a memory and a processor coupled to the memory, the processor being configured to perform:
obtaining an image to be inspected, the image to be inspected comprising a sequence of slice images;
determining a corresponding group of slice images for each target image in the sequence of slice images, the group of slice images comprising the target image and an adjacent image having a context relationship with the target image in the sequence of slice images;
extracting a corresponding slice feature map for each slice image in the group of slice images;
aligning the slice feature maps extracted corresponding to the group of slice images;
aggregating context information of each slice image in the group of slice images by using an aligned feature map; and
performing target region inspection on an aggregated feature map, to obtain an inspection result corresponding to the target image, and combining the inspection result corresponding to each target image, to generate an inspection result corresponding to the image to be inspected.