# US 11,908,580 B2 · Image classification method, computer-readable storage medium, and computer device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,580 B2
- Document ID: US11908580B2
- Status: Granted
- Kind: B2
- Application number: 17/397,857
- Filed: 2021-08-09
- Published: 2021-11-25
- Granted: 2024-02-20
- Earliest claimed: 2019-05-08
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Yifan Hu; Yefeng Zheng
- Web page: https://patentgrep.com/patent/US11908580B2

## CPC

- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 18/2413: based on distances to training or reference patterns
- G06F 18/253: {of extracted features}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/776: Validation; Performance evaluation
- G06V 10/806: {of extracted features}
- G16H 30/40: for processing medical images, e.g. editing
- G06N 3/08: Learning methods
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A computer device obtains a plurality of medical images. The device generates a texture image based on image data of a region of interest in the medical images. The device extracts a local feature from the texture image using a first network model. The device extracts a global feature from the medical images using a second network model. The device fuses the extracted local feature and the extracted global feature to form a fused feature. The device performs image classification based on the fused feature.

## Claims

### Claim 1

An image classification method, applicable to a computer device, the method comprising:
obtaining a plurality of medical images;
generating a texture image based on image data of a region of interest (ROI) in the medical images;
extracting a local feature from the texture image using a first network model;
extracting a global feature from the medical images using a second network model;
fusing the extracted local feature and the extracted global feature to form a fused feature using a fully connected layer;
performing convolution calculation on the fused feature; and
performing image classification on the medical images based on a result obtained through the convolution calculation of the fused feature.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising obtaining the ROI in the medical images, the obtaining including:
obtaining medical image samples having segmentation labels;
inputting the medical image samples to an image segmentation network for segmentation to obtain a predicted region of interest; and
inputting the obtained medical images to a trained image segmentation network for image segmentation when the predicted region of interest matches the segmentation labels, to obtain the region of interest of the medical images.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising obtaining the ROI in the medical images, the obtaining including:
receiving an input that defines the ROI in the medical images and defining the ROI in accordance with the input; or
obtaining pixel values of the medical images, and obtaining, when pixel values meet a target condition, a region corresponding to the pixel values meeting the target condition as the ROI.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the generating the texture image further comprises:
obtaining a texture feature matrix based on the image data of the ROI;
obtaining a target size based on a size of the texture feature matrix; and
performing image rendering based on the texture feature matrix to obtain the texture image having the target size.

### Claim 5 (depends on 4)

The method according to claim 4, wherein obtaining the texture feature matrix further comprises:
selecting a pixel value interval;
obtaining pixel combinations of pixel values in the pixel value interval;
calculating frequencies at which pixels corresponding to the pixel values in the pixel combinations appear in the ROI to obtain frequency sets; and
converting frequencies in the frequency sets into probabilities to obtain the texture feature matrix.

### Claim 6 (depends on 5)

The method according to claim 5, wherein calculating the frequencies further comprises:
obtaining a target distance and a calculation direction, there being a plurality of calculation directions;
obtaining, from the region of interest, a pixel that matches the target distance;
calculating, based on each of the calculation directions, a number of pixels that correspond to pixel values in each of the pixel combinations and that match the pixel meeting the target distance; and
obtaining a matching quantity as the frequencies to obtain the plurality of frequency sets corresponding to a quantity of the calculation directions.

### Claim 7 (depends on 6)

The method according to claim 6, wherein converting the frequencies in the frequency sets into probabilities to obtain the texture feature matrix comprises:
obtaining a quantity of channels of the medical images;
converting the frequencies in the plurality of frequency sets into the probabilities, and calculating a mean of probabilities at corresponding positions in the plurality of frequency sets after the conversion;
obtaining, based on the mean of the probabilities, mean probability sets with a quantity consistent with the quantity of channels; and
obtaining the mean probability sets as the texture feature matrix.

### Claim 8 (depends on 1)

The method according to claim 1, further comprising:
obtaining medical image samples of different image types and a corresponding reference category;
generating a training texture image based on data of a region of interest in the medical image samples;
extracting a feature of the training texture image using the first network model, to obtain a local training medical feature;
extracting features of the medical image samples using the second network model, to obtain a global training medical feature;
fusing the global training medical feature and the local training medical feature by using the fully connected layer;
performing image classification based on a fused feature obtained after the fusion to obtain a classification result; and
respectively adjusting parameter values of the first network model, the second network model, and the fully connected layer based on an error between the classification result and the reference category.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising:
performing image processing on the medical image samples different the different image types to obtain a plurality of groups of medical image samples; and
generating the training texture image further comprises:
extracting a training texture feature matrix based on the data of the region of interest in the plurality of groups of medical image samples; and
generating the training texture image based on the training texture feature matrix.

### Claim 10 (depends on 8)

The method according to claim 8, wherein the respectively adjusting parameter values of the first network model, the second network model, and the fully connected layer based on an error between the classification result and the reference category further comprises:
determining the error between the classification result and the reference category;
back-propagating the error to neurons of the first network model, the second network model, and the fully connected layer to obtain a gradient of parameter values of the neurons; and
updating the parameter values of the neurons based on the gradient.

### Claim 11

A computer device, comprising:
one or more processors; and
memory storing one or more programs, that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
obtaining a plurality of medical images;
generating a texture image based on image data of a region of interest (ROI) in the medical images;
extracting a local feature from the texture image using a first network model;
extracting a global feature from the medical images using a second network model;
fusing the extracted local feature and the extracted global feature to form a fused feature using a fully connected layer;
Performing convolution calculation on the fused feature; and
performing image classification on the medical images based on a result obtained through the convolution calculation of the fused feature.

### Claim 12 (depends on 11)

The computer device according to claim 11, the operations further comprising obtaining the ROI in the medical images, the obtaining including:
obtaining medical image samples having segmentation labels;
inputting the medical image samples to an image segmentation network for segmentation to obtain a predicted region of interest; and
inputting the obtained medical images to a trained image segmentation network for image segmentation when the predicted region of interest matches the segmentation labels, to obtain the region of interest of the medical images.

### Claim 13 (depends on 11)

The computer device according to claim 11, the operations further comprising obtaining the ROI in the medical images, the obtaining including:
receiving an input that defines the ROI in the medical images and defining the ROI in accordance with the input; or
obtaining pixel values of the medical images, and obtaining, when pixel values meet a target condition, a region corresponding to the pixel values meeting the target condition as the ROI.

### Claim 14 (depends on 11)

The computer device according to claim 11, wherein generating the texture image further comprises:
obtaining a texture feature matrix based on the image data of the ROI;
obtaining a target size based on a size of the texture feature matrix; and
performing image rendering based on the texture feature matrix to obtain the texture image having the target size.

### Claim 15 (depends on 14)

The computer device according to claim 14, wherein obtaining the texture feature matrix further comprises:
selecting a pixel value interval;
obtaining pixel combinations of pixel values in the pixel value interval;
calculating frequencies at which pixels corresponding to the pixel values in the pixel combinations appear in the ROI to obtain frequency sets;
converting frequencies in the frequency sets into probabilities to obtain the texture feature matrix.

### Claim 16 (depends on 15)

The computer device according to claim 15, wherein calculating the frequencies further comprises:
obtaining a target distance and a calculation direction, there being a plurality of calculation directions;
obtaining, from the region of interest, a pixel that matches the target distance;
calculating, based on each of the calculation directions, a number of pixels that correspond to pixel values in each of the pixel combinations and that match the pixel meeting the target distance; and
obtaining a matching quantity as the frequencies to obtain the plurality of frequency sets corresponding to a quantity of the calculation directions.

### Claim 17

A non-transitory computer readable storage medium storing instructions that, when executed by one or more processors of a computer device, cause the one or more processors to perform operations comprising:
obtaining a plurality of medical images;
generating a texture image based on image data of a region of interest (ROI) in the medical images;
extracting a local feature from the texture image using a first network model;
extracting a global feature from the medical images using a second network model;
fusing the extracted local feature and the extracted global feature to form a fused feature using a fully connected layer;
performing convolution calculation on the fused feature; and
performing image classification on the medical images based on a result obtained through the convolution calculation of the fused feature.

### Claim 18 (depends on 17)

The non-transitory computer readable storage medium according to claim 17, the operations further comprising obtaining the ROI in the medical images, the obtaining including:
obtaining medical image samples having segmentation labels;
inputting the medical image samples to an image segmentation network for segmentation to obtain a predicted region of interest; and
inputting the obtained medical images to a trained image segmentation network for image segmentation when the predicted region of interest matches the segmentation labels, to obtain the region of interest of the medical images.

### Claim 19 (depends on 17)

The non-transitory computer readable storage medium according to claim 17, the operations further comprising obtaining the ROI in the medical images, the obtaining including:
receiving an input that defines the ROI in the medical images and defining the ROI in accordance with the input; or
obtaining pixel values of the medical images, and obtaining, when pixel values meet a target condition, a region corresponding to the pixel values meeting the target condition as the ROI.