# US 12,217,869 B2 · Image diagnosis apparatus using deep learning model and method therefor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,869 B2
- Document ID: US12217869B2
- Status: Granted
- Kind: B2
- Application number: 17/624,621
- Filed: 2020-06-30
- Published: 2022-08-25
- Granted: 2025-02-04
- Earliest claimed: 2019-07-05
- Assignees: Korea Advanced Institute of Science and Technology; Seegene Medical Foundation
- Applicants: Korea Advanced Institute of Science and Technology; Seegene Medical Foundation
- Inventors: Mun Yong Yi; Young Jin Park; Jong Kee Chun; Young Sin Ko
- Web page: https://patentgrep.com/patent/US12217869B2

## CPC

- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06T 7/0012: {Biomedical image inspection}
- G06V 10/82: using neural networks
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30004: Biomedical image processing
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present specification discloses an image diagnosis apparatus using a deep learning model and a method therefor, wherein tissue included in an input medical image is classified as being one of normal and abnormal for a disease by using the trained deep learning model using a weighted loss function in which different weights are assigned to a probability distribution of determining that a picture extracted from the input medical image is abnormal even though it is normal and a probability distribution of determining that the picture is normal even though it is abnormal.

## Claims

### Claim 1

An image diagnosis apparatus using a deep learning model, comprising:
an image input part receiving a medical image including tissue of a human body;
a classifier classifying the tissue included in the medical image as being one of normal and abnormal in terms of the presence of disease using a trained deep learning model using a weighted loss function in which different weights are assigned to a probability distribution of determining that a feature extracted from the input medical image is abnormal even though the feature is normal and a probability distribution of determining that the feature is normal even though the feature is abnormal; and
a result output part outputting a result of classification by the classifier.

### Claim 2 (depends on 1)

The image diagnosis apparatus of claim 1, wherein the deep model learning includes a feature extraction part extracting the feature of the medical image by performing convolution and subsampling on the input medical image, and a classification part classifying the input medical image using the weighted loss function with respect to the extracted feature.

### Claim 3 (depends on 1)

The image diagnosis apparatus of claim 1, wherein, in the weighted loss function, a second weight assigned to the probability distribution of determining that the feature is normal even though the feature is abnormal is greater than a first weight assigned to the probability distribution of determining that the feature is abnormal even though the feature is normal.

### Claim 4 (depends on 3)

The image diagnosis apparatus of claim 3, wherein the number of the deep learning model is two or more, and
the result output part ensembles classification results of the two or more deep learning models and classifies the tissue included in the medical image as being one of normal and abnormal in terms of the presence of disease.

### Claim 5 (depends on 4)

The image diagnosis apparatus of claim 4, wherein all or part of the deep learning models applies a dropout technique that learns training data by removing a node present in each layer with a specific probability and, when the learning is completed and a test is conducted, considers all nodes as they are and multiplies the probability by a weight of each node.

### Claim 6 (depends on 4)

The image diagnosis apparatus of claim 4, wherein at least part of the deep learning models use a same single deep learning model and are trained by adjusting a number or order of pieces of training data used for training the single deep learning model.

### Claim 7 (depends on 4)

The image diagnosis apparatus of claim 4, wherein at least part of the deep learning models are different from deep learning models having different layer depths or a different number of layers.

### Claim 8 (depends on 4)

The image diagnosis apparatus of claim 4, wherein the result output part ensembles classification results of the two or more deep learning models based on at least one of a majority vote-based ensemble, a unanimity-based ensemble, or an uncertainty-based ensemble, and classifies the tissue included in the medical image as being one of normal and abnormal in terms of the presence of disease.

### Claim 9 (depends on 8)

The image diagnosis apparatus of claim 8, wherein the uncertainty-based ensemble ensembles the classification results of the two or more deep learning models using either a confidence average or a standard deviation.

### Claim 10 (depends on 9)

The image diagnosis apparatus of claim 9, wherein the medical image is an image captured from large intestine of the human body, and the disease is colon cancer.

### Claim 11

An image diagnosis method using a deep learning model, comprising:
an image input operation of receiving a medical image including tissue of a human body;
a classification operation of classifying the tissue included in the input medical image as being one of normal and abnormal in terms of the presence of disease using a trained deep learning model using a weighted loss function in which different weights are assigned to a probability distribution of determining that a feature extracted from the input medical image is abnormal even though the feature is normal and a probability distribution of determining that the feature is normal even though the feature is abnormal; and
a result output operation of outputting a result of classification in the classification operation.

### Claim 12 (depends on 11)

The image diagnosis method of claim 11, wherein the deep learning model includes a feature extraction part configured to extract the feature of the medical image by performing convolution and subsampling on the input medical image, and a classification part configured to classify the input medical image using the weighted loss function with respect to the extracted feature.

### Claim 13 (depends on 11)

The image diagnosis apparatus of claim 11, wherein, in the weighted loss function, a second weight assigned to the probability distribution of determining that the feature is normal even though the feature is abnormal is greater than a first weight assigned to the probability distribution of determining that the feature is abnormal even though the feature is normal.

### Claim 14 (depends on 13)

The image diagnosis method of claim 13, wherein the number of the deep learning model is two or more, and
in the result output operation, classification results of the two or more deep learning models are ensembled and the tissue included in the medical image is classified as being one of normal and abnormal in terms of the presence of disease.

### Claim 15 (depends on 14)

The image diagnosis method of claim 14, wherein all or part of the deep learning models applies a dropout technique that learns training data by removing a node present in each layer with a specific probability and, when the learning is completed and a test is conducted, considers all nodes as they are and multiplies the probability by a weight of each node.

### Claim 16 (depends on 14)

The image diagnosis method of claim 14, wherein at least part of the deep learning models use a same single deep learning model and are trained by adjusting a number or order of pieces of training data used for training the single deep learning model.

### Claim 17 (depends on 14)

The image diagnosis method of claim 14, wherein at least part of the deep learning models are different from deep learning models having different layer depths or a different number of layers.

### Claim 18 (depends on 14)

The image diagnosis method of claim 14, wherein in the result output operation, the classification results of the two or more deep learning models are ensembled based on at least one of a majority vote-based ensemble, a unanimity-based ensemble, or an uncertainty-based ensemble, and the tissue included in the medical image is classified as being one of normal and abnormal in terms of the presence of disease.

### Claim 19 (depends on 18)

The image diagnosis method of claim 18, wherein the uncertainty-based ensemble ensembles the classification results of the two or more deep learning models using either a confidence average or a standard deviation.

### Claim 20 (depends on 19)

The image diagnosis method of claim 19, wherein the medical image is an image captured from large intestine of the human body, and the disease is colon cancer.