# US 2024/0188797 A1 · Method, apparatus and system for providing medical diagnosis assistance information based on artificial intelligence

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0188797 A1
- Document ID: US20240188797A1
- Status: Published application
- Kind: A1
- Application number: 18/533,869
- Filed: 2023-12-08
- Published: 2024-06-13
- Earliest claimed: 2022-12-09
- Assignees: VPIX Medical Incorporation
- Applicants: VPIX Medical Incorporation
- Inventors: Kyuyoung Kim; Kyungmin Hwang
- Web page: https://patentgrep.com/patent/US20240188797A1

## CPC

- A61B 1/000096: {using artificial intelligence}
- A61B 1/043: {for fluorescence imaging}
- A61B 1/063: {for monochromatic or narrow-band illumination}
- A61B 1/0653: {with wavelength conversion}
- A61B 5/0062: {Arrangements for scanning}
- A61B 5/0071: {by measuring fluorescence emission}
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/11: Region-based segmentation
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10064: Fluorescence image
- G06T 2207/10068: Endoscopic image
- G06T 2207/20081: Training; Learning
- G06T 2207/30016: Brain
- G06T 2207/30061: Lung
- G06T 2207/30081: Prostate
- G06T 2207/30088: Skin; Dermal
- G06T 2207/30096: Tumor; Lesion
- G06T 2207/30101: Blood vessel; Artery; Vein; Vascular
- G06T 2210/41: Medical
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Proposed is a method, apparatus, and system for providing medical diagnosis assistance information showing whether a tissue related to a tissue image is normal or abnormal (e.g., cancer) using the AI technology. Proposed are a method, an apparatus, and a system for training an AI model for producing medical diagnosis assistance information in consideration of conditions for obtaining a tissue image in an endomicroscope system for digital biopsy.

## Claims

### Claim 1

A confocal fluorescence endomicroscope system comprising:
a laser scanner;
a light receiver; and
a processor configured to:
generate a first driving signal corresponding to a first axis and a second driving signal corresponding to a second axis different from the first axis;
generate a first image to detect excitation light generated by light emitted to a tissue to be examined from an end of the laser scanner operating based on the first driving signal and the second driving signal;
obtain medical diagnosis assistance information including information on whether a tissue related to the first image is normal from an artificial intelligence model by inputting the first image into the artificial intelligence model;
identify an activated region related to the medical diagnosis assistance information on the first image;
determine that a condition for providing the medical diagnosis assistance information is satisfied in response to a size of the activated region at a specific color threshold being a threshold or more in an entire region and the number of closed loops of the activated region at the specific color threshold being less than or equal to a predetermined number;
provide the medical diagnosis assistance information on the first image when the condition is satisfied, and
do not provide the medical diagnosis assistance information on the first image in response to the condition being not satisfied,
wherein the specific color threshold is a pre-designated value, or a value selected by a user, and
wherein the artificial intelligence model is trained based on a wavelength value of light, organ information of a tissue, and a plurality of images related to a tissue.

### Claim 2 (depends on 1)

The confocal fluorescence endomicroscope system of claim 1, wherein the processor is configured to:
obtain new medical diagnosis assistance information by inputting again the first image into the artificial intelligence model,
identify an activated region related to the new medical diagnosis assistance information, and
determine again whether the condition is satisfied, in response to the condition being not satisfied.

### Claim 3 (depends on 1)

The confocal fluorescence endomicroscope system of claim 1, wherein the processor is configured to output the first image including the medical diagnosis assistance information to an external device in response to the condition being satisfied.

### Claim 4 (depends on 1)

The confocal fluorescence endomicroscope system of claim 1, wherein the light includes first light having a first wavelength and second light having a second wavelength, and
wherein the processor is configured to:
generate a first assistant image about the tissue using the first light and generate a second assistant image about the tissue using the second light;
generate the first image by combining the first assistant image and the second assistant image; and
determine final medical diagnosis assistance information on the tissue using first medical diagnosis assistance information obtained by inputting the first assistant image into the artificial intelligence model, second medical diagnosis assistance information obtained by inputting the second assistant image into the artificial intelligence model, and third medical diagnosis assistance information obtained by inputting the first image into the artificial intelligence model.

### Claim 5 (depends on 4)

The confocal fluorescence endomicroscope system of claim 4, wherein the first wavelength is included in one wavelength band of 400 nm to 410 nm, 480 nm to 500 nm, or 770 nm to 880 nm, and
wherein the second wavelength is included in another wavelength band different from the wavelength band including the first wavelength.

### Claim 6 (depends on 1)

The confocal fluorescence endomicroscope system of claim 1, wherein the processor is configured to:
identify the organ information about the tissue;
identify the number of specific images corresponding to the organ information; and
determine medical diagnosis assistance information about the tissue using a plurality of images of the tissue in response to the plurality of images being obtained by the number of the specific images.

### Claim 7 (depends on 6)

The confocal fluorescence endomicroscope system of claim 6, wherein the processor is configured to:
identify medical diagnosis assistance information on each of the plurality of images in response to the plurality of images being obtained by the number of the specific images while moving a probe including the laser scanner along a predetermined pattern; and
output a message that ignores medical diagnosis assistance information on one or more images, which relate to first type of medical diagnosis assistance information of the medical diagnosis assistance information about each of the plurality of images, or requests new medical diagnosis assistance information about the one or more images, in response to the number of the one or more images being a predetermined number or less and a minimum distance between the one or more images exceeds a predetermined distance.

### Claim 8

A method of providing medical diagnosis assistance information using a confocal fluorescence endomicroscope system, the method comprising:
generating a first driving signal corresponding to a first axis and a second driving signal corresponding to a second axis different from the first axis;
generating a first image to detect excitation light generated by light emitted to a tissue to be examined from an end of a laser scanner operating based on the first driving signal and the second driving signal;
inputting the first image into an artificial intelligence model;
obtaining medical diagnosis assistance information including information on whether a tissue related to the first image is normal from the artificial intelligence model;
identifying an activated region related to the medical diagnosis assistance information on the first image;
determining that a condition for providing the medical diagnosis assistance information is satisfied in response to a size of the activated region at a specific color threshold being a threshold or more in an entire region and the number of closed loops of the activated region at the specific color threshold being less than or equal to a predetermined number;
providing the medical diagnosis assistance information on the first image when the condition is satisfied; and
not providing the medical diagnosis assistance information on the first image in response to the condition being not satisfied,
wherein the specific color threshold is a pre-designated value or a value selected by a user, and
wherein the artificial intelligence model is trained based on a wavelength value of light, organ information of a tissue, and a plurality of images related to a tissue.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
in response to the condition being not satisfied,
obtaining new medical diagnosis assistance information by inputting again the first image into the artificial intelligence model;
identifying an activated region related to the new medical diagnosis assistance information; and
determining again whether the condition is satisfied.

### Claim 10 (depends on 8)

The method of claim 8, further comprising outputting the first image including the medical diagnosis assistance information to an external device when the condition is satisfied.

### Claim 11 (depends on 8)

The method of claim 8, wherein the light includes first light having a first wavelength and second light having a second wavelength, and
the method further comprising:
generating a first assistant image about the tissue using the first light and generate a second assistant image about the tissue using the second light;
generating the first image by combining the first assistant image and the second assistant image; and
determining final medical diagnosis assistance information on the tissue using first medical diagnosis assistance information obtained by inputting the first assistant image into the artificial intelligence model, second medical diagnosis assistance information obtained by inputting the second assistant image into the artificial intelligence model, and third medical diagnosis assistance information obtained by inputting the first image into the artificial intelligence model.

### Claim 12 (depends on 11)

The method of claim 11, wherein the first wavelength is included in one wavelength band of 400 nm to 410 nm, 480 nm to 500 nm, or 770 nm to 880 nm, and
wherein the second wavelength is included in another wavelength band different from the wavelength band including the first wavelength.

### Claim 13 (depends on 8)

The method of claim 8, further comprising:
identifying the organ information about the tissue;
identifying the number of specific images corresponding to the organ information; and
determining medical diagnosis assistance information on the tissue using a plurality of images of the tissue in response to the plurality of images being obtained by the number of the specific images.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
identifying medical diagnosis assistance information on each of the plurality of images in response to the plurality of images being obtained by the number of the specific images while moving a probe including the laser scanner along a predetermined pattern; and
outputting a message that ignores medical diagnosis assistance information on one or more images, which relate to first type of medical diagnosis assistance information of the medical diagnosis assistance information about each of the plurality of images, or requests new medical diagnosis assistance information about the one or more images, in response to the number of the one or more images being a predetermined number or less and a minimum distance between the one or more images exceeds a predetermined distance.
