# US 12,593,957 B2 · Endoscopic examination support apparatus, endoscopic examination support method, and recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,593,957 B2
- Document ID: US12593957B2
- Status: Granted
- Kind: B2
- Application number: 18/530,375
- Filed: 2023-12-06
- Published: 2024-06-06
- Granted: 2026-04-07
- Earliest claimed: 2023-12-06
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Naoto Maeda; Takuma Igarashi; Masahiro Saikou; Kimiyasu Takoh; Yuko Maekawa
- Web page: https://patentgrep.com/patent/US12593957B2

## CPC

- A61B 1/045: Control thereof
- A61B 1/000094: {extracting biological structures}
- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/66: of image moments or centre of gravity
- G06T 11/60: Creating or editing images; Combining images with text
- G16H 30/40: for processing medical images, e.g. editing
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/10068: Endoscopic image
- G06T 2207/20081: Training; Learning
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30096: Tumor; Lesion
- G06T 2210/41: Medical
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

In the endoscopic examination support apparatus, the video acquisition means acquires an endoscopic video captured by an endoscope. The lesion detection means detects a lesion candidate from the endoscopic video. The heat map generation means generates a heat map which represents a lesion possibility of the lesion candidate included in the endoscopic video by a color. The display control means superimposes and displays an area where the lesion possibility is equal to or higher than a first threshold on the endoscopic video in a first display mode, and superimposes and displays an area where the lesion possibility is equal to or higher than a second threshold on the endoscopic video in a second display mode.

## Claims

### Claim 1

An endoscopic examination support apparatus comprising:
a memory configured to store instructions; and
a processor configured to execute the instructions to:
acquire an endoscopic video captured by an endoscope;
detect a lesion candidate from the endoscopic video;
for each pixel of a frame including the lesion candidate, estimate a lesion score, indicating a probability that the pixel belongs to a lesion area, using a lesion-score estimation model learned in advance, and generate a heat map by converting the lesion score into colors;
extract, from the heat map, a first threshold area consisting of first pixels having the lesion score equal to or greater than a first threshold and a second threshold area consisting of second pixels having the lesion score equal to or greater than a second threshold higher than the first threshold;
superimpose and display, in a first display mode, the first threshold area on the endoscopic video; and
superimpose and display, in a second display mode, the second threshold area on the endoscopic video, the second display mode being set to a lower transparency than is the first display mode,
wherein superimposing and displaying any of the first threshold area on the endoscopic video in the first display mode and the second threshold area on the endoscopic video in the second display mode is based on the processor determining that the lesion candidate has been obtained from a predetermined number of consecutive frames of the endoscopic video.

### Claim 2 (depends on 1)

The endoscopic examination support apparatus according to claim 1, wherein a degree of transparency of any of the first display mode and the second display mode depends on the lesion score.

### Claim 3 (depends on 1)

The endoscopic examination support apparatus according to claim 1, wherein a predetermined mark is superimposed and displayed at a position of any of a center and a center of gravity of an area as having a determined highest lesion possibility.

### Claim 4 (depends on 1)

The endoscopic examination support apparatus according to claim 1, wherein the processor is further configured to execute the instructions to:
detect the lesion candidate by using a machine learning model learned in advance to estimate the lesion candidate included in the endoscopic video.

### Claim 5 (depends on 3)

The endoscopic examination support apparatus according to claim 3, wherein the predetermined mark is displayed as a guide to a position of the lesion candidate.

### Claim 6

An endoscopic examination support method comprising:
acquiring an endoscopic video captured by an endoscope;
detecting a lesion candidate from the endoscopic video;
for each pixel of a frame including the lesion candidate, estimate a lesion score, indicating a probability that the pixel belongs to a lesion area, using a lesion-score estimation model learned in advance, and generate a heat map by converting the lesion score into colors;
extracting, from the heat map, a first threshold area consisting of first pixels having the lesion score equal to or greater than a first threshold and a second threshold area consisting of second pixels having the lesion score equal to or greater than a second threshold higher than the first threshold;
superimposing and displaying, in a first display mode, the first threshold area on the endoscopic video; and
superimpose and display, in a second display mode, the second threshold area on the endoscopic video, the second display mode being set to a lower transparency than is the first display mode,
wherein superimposing and displaying any of the first threshold area on the endoscopic video in the first display mode and the second threshold area on the endoscopic video in the second display mode is based on determining that the lesion candidate has been obtained from a predetermined number of consecutive frames of the endoscopic video.

### Claim 7

A non-transitory computer-readable recording medium recording a program, the program causing a computer to execute:
acquiring an endoscopic video captured by an endoscope;
detecting a lesion candidate from the endoscopic video;
for each pixel of a frame including the lesion candidate, estimate a lesion score, indicating a probability that the pixel belongs to a lesion area, using a lesion-score estimation model learned in advance, and generate a heat map by converting the lesion score into colors;
extracting, from the heat map, a first threshold area consisting of first pixels having the lesion score equal to or greater than a first threshold and a second threshold area consisting of second pixels having the lesion score equal to or greater than a second threshold higher than the first threshold;
superimposing and displaying, in a first display mode, the first threshold area on the endoscopic video; and
superimpose and display, in a second display mode, the second threshold area on the endoscopic video, the second display mode being set to a lower transparency than is the first display mode,
wherein superimposing and displaying any of the first threshold area on the endoscopic video in the first display mode and the second threshold area on the endoscopic video in the second display mode is based on determining that the lesion candidate has been obtained from a predetermined number of consecutive frames of the endoscopic video.