# US 12,119,116 B2 · Apparatus, information processing method, and computer-readable recording medium for disease correlation and inferencing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,119,116 B2
- Document ID: US12119116B2
- Status: Granted
- Kind: B2
- Application number: 18/562,901
- Filed: 2022-09-26
- Published: 2024-07-11
- Granted: 2024-10-15
- Earliest claimed: 2021-09-30
- Assignees: Deepeyevision Inc.
- Applicants: Deepeyevision Inc.
- Inventors: Yusuke Kondo; Hidenori Takahashi
- Web page: https://patentgrep.com/patent/US12119116B2

## CPC

- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- A61B 3/10: Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- G06N 3/08: Learning methods
- G06T 7/00: Image analysis
- G06T 7/0012: {Biomedical image inspection}
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06T 2207/30096: Tumor; Lesion
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An information processing apparatus that comprises a processor configured to: acquire an image acquisition unit that acquires a medical image; extract one or more image feature values from the medical image; infer one or more diseases predicted from the medical image based on the one or more image feature values and a classification basis, the classification basis being a set of C basis vectors with each basis vector representing correlations between a disease of a set of C diseases and other diseases by using a vector of a certain dimension, wherein C is an integer of 2 or more; and output results identifying the one or more inferred diseases.

## Claims

### Claim 1

An information processing apparatus comprising:
a processor configured to:
acquire a medical image;
extract one or more image feature values from the medical image;
infer one or more diseases predicted from the medical image based on the one or more image feature values and a classification basis, the classification basis being a set of C basis vectors with each basis vector representing correlations between a disease of a set of C diseases and other diseases by using a vector of a certain dimension, wherein C is an integer of 2 or more; and
output results identifying the one or more inferred diseases.

### Claim 2 (depends on 1)

The information processing apparatus according to claim 1,
wherein the
processor is further configured to apply an activation function to a Hadamard product of the one or more image feature values and the classification basis to obtain respective certainty factors of the set of C diseases.

### Claim 3 (depends on 1)

The information processing apparatus according to claim 1,
wherein the classification basis is obtained by causing a second machine learning model to do learning, where the second machine learning model accepts data embedded with label information about a plurality of medical images labeled with one or more diseases as input data, and outputs a feature that has learned correlations among diseases.

### Claim 4 (depends on 3)

The information processing apparatus according to claim 3,
wherein the data accepted as the input data is graph data embedded with the label information about the plurality of medical images labeled with one or more diseases, the label information being in multi-hot vector form, and
the second machine learning model is a graph convolutional neural network, with global max pooling (GMP) being applied in a final convolutional layer.

### Claim 5 (depends on 4)

The information processing apparatus according to claim 4,
wherein the graph data includes a node corresponding to a disease or a lesion, and
wherein one or more lesions are inferred in addition to the one or more diseases from the medical images.

### Claim 6 (depends on 2)

The information processing apparatus according to claim 2, wherein the processor is further configured to, when a certainty factor of each of the diseases obtained through inference is larger than a certain threshold, acquire such diseases as one or more candidate diseases.

### Claim 7 (depends on 6)

The information processing apparatus according to claim 6, wherein the threshold is set for each disease, for a group including a plurality of diseases, or for a combination of each disease and the group.

### Claim 8 (depends on 7)

The information processing apparatus according to claim 7, wherein the threshold is set so as to maximize a certain evaluation criterion for the certainty factor of each of the diseases obtained through inference.

### Claim 9 (depends on 7)

The information processing apparatus according to claim 7,
wherein the one or more image feature values are acquired by inputting the medical image to a first learned model,
the first learned model is a learned model that has learned to estimate one or more image feature values from a medical image, and
wherein the threshold is set so as to maximize prediction accuracy of inferencing, the prediction accuracy being evaluated by k-fold cross-validation of learning data used in generating the first learned model.

### Claim 10

A method comprising:
acquiring a medical image;
extracting one or more image feature values from the medical image;
inferring one or more diseases predicted from the medical image, based on the one or more image feature values and a classification basis, the classification basis being a set of C basis vectors with each basis vector representing correlations between a disease of a set of C diseases and other diseases by using a vector of a certain dimension, wherein C is an integer of 2 or more; and
outputting inferred results.

### Claim 11

A non-transitory computer-readable recording medium recording a program that causes one or more computers to perform the processes of:
acquiring a medical image;
extracting one or more image feature values from the medical image;
inferring one or more diseases predicted from the medical image, based on the one or more image feature values and a classification basis, the classification basis being a set of C basis vectors with each basis vector representing correlations between a disease of a set of C diseases and other diseases by using a vector of a certain dimension, wherein C is an integer of 2 or more; and
outputting inferred results.

### Claim 12

An information processing apparatus comprising:
a processor configured to:
generate a first machine learning model that outputs one or more image feature values from an image, the image feature value representing a feature of the image;
generate a second machine learning model that outputs a classification basis that has learned correlations among diseases from input data generated based on label information about a plurality of medical images labeled with one or more diseases;
cause the first machine learning model and the second machine learning model to do learning using the plurality of medical images labeled with one or more diseases, find certainty factors of the respective diseases based on the one or more image feature values acquired from the first machine learning model and the classification basis acquired from the second machine learning model, and cause the first machine learning model and the second machine learning model to do learning so as to minimize differences between the certainty factors and the diseases labeled to the medical images, the classification basis being a set of C basis vectors with each basis vector representing correlations between a disease of a set of C diseases and other diseases by using a vector of a certain dimension, wherein C is an integer of 2 or more;
output a learned model obtained through learning done by the first machine learning model; and
output the classification basis obtained through learning done by the second machine learning model.

### Claim 13

A method comprising:
acquiring a plurality of medical images labeled with one or more diseases and a plurality of medical images free of diseases;
causing a first machine learning model and a second machine learning model to do learning using the medical images, the first machine learning model being a machine learning model that outputs one or more image feature values from an image, the image feature value representing a feature of the image, the second machine learning model being a machine learning model that outputs a classification basis that has learned correlations among diseases from input data generated based on label information about the plurality of medical images labeled with one or more diseases, finding certainty factors of the respective diseases based on the one or more image feature values acquired from the first machine learning model and the classification basis acquired from the second machine learning model, and causing the first machine learning model and the second machine learning model to do learning so as to minimize differences between the certainty factors and the diseases labeled to the medical images, the classification basis being a set of C basis vectors with each basis vector representing correlations between a disease of a set of C diseases and other diseases by using a vector of a certain dimension, wherein C is an integer of 2 or more;
outputting a learned model obtained through learning done by the first machine learning model; and
outputting the classification basis obtained through learning done by the second machine learning model.