# US 2023/0410334 A1 · Endoscopic image processing apparatus and endoscopic image processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410334 A1
- Document ID: US20230410334A1
- Status: Published application
- Kind: A1
- Application number: 18/236,572
- Filed: 2023-08-22
- Published: 2023-12-21
- Earliest claimed: 2021-02-25
- Assignees: Olympus Medical Systems Corp.
- Applicants: Olympus Medical Systems Corp.
- Inventors: Hirokazu NISHIMURA
- Web page: https://patentgrep.com/patent/US20230410334A1

## CPC

- A61B 1/000094: {extracting biological structures}
- G06T 7/50: Depth or shape recovery
- A61B 1/00009: {of image signals during a use of endoscope}
- A61B 1/000096: {using artificial intelligence}
- G06T 7/11: Region-based segmentation
- G06T 2207/10068: Endoscopic image
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30028: Colon; Small intestine
- H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image acquisition unit acquires an endoscopic image photographed by an endoscope. A segmentation unit partitions the endoscopic image into a plurality of regions. A depth information generator generates depth information indicating a depth of the endoscopic image. A recognition unit specifies a direction in which the endoscope is advanceable based on region information indicating a result of segmentation by the segmentation unit and depth information of the endoscopic image generated by the depth information generator.

## Claims

### Claim 1

An endoscopic image processing apparatus comprising one or more processors having hardware, wherein
the one or more processors are configured to:
acquire an endoscopic image photographed by an endoscope;
partition the acquired endoscopic image into a plurality of regions;
generate depth information indicating a depth of the acquired endoscopic image; and
specify a direction in which the endoscope is advanceable based on region information indicating a result of segmentation and the depth information of the endoscopic image.

### Claim 2 (depends on 1)

The endoscopic image processing apparatus according to claim 1, wherein
the one or more processors are configured to:
partition the endoscopic image into regions corresponding to a plurality of structures, and
one of the regions of the plurality of structures is a region in which the endoscope is advanceable.

### Claim 3 (depends on 2)

The endoscopic image processing apparatus according to claim 2, wherein
the one or more processors are configured to:
generate information on an advancing direction of the endoscope based on the direction in which the endoscope is advanceable and a direction in which the endoscope should not be advanced.

### Claim 4

An endoscopic image processing apparatus, comprising one or more processors having hardware, wherein
the one or more processors are configured to:
acquire an endoscopic image photographed by an endoscope;
select one or more operation contents from a plurality of predetermined operation contents based on the acquired endoscopic image;
partition the acquired endoscopic image into a plurality of regions;
recognize a situation around a distal end portion of the endoscope based on region information indicating a result of segmentation; and
determine an operation content to be performed based on the selected one or more operation contents and the recognized situation.

### Claim 5 (depends on 4)

The endoscopic image processing apparatus according to claim 4, wherein
the one or more processors are configured to:
determine whether or not the selected one or more operation contents are appropriate based on the recognized situation.

### Claim 6 (depends on 4)

The endoscopic image processing apparatus according to claim 4, wherein
the one or more processors are configured to:
select the one or more operation contents by inputting input data acquired from the acquired endoscopic image to one or more operation selection models generated by machine learning that uses, as supervised data, learning images that are endoscopic images photographed in the past and labels that indicate operation contents for the endoscopes that have photographed the learning images.

### Claim 7 (depends on 4)

The endoscopic image processing apparatus according to claim 4, wherein
the one or more processors are configured to:
generate depth information indicating a depth of the acquired endoscopic image, and
specify a direction in which the endoscope is advanceable based on region information indicating a result of segmentation and the depth information of the endoscopic image.

### Claim 8

A method of processing an endoscopic image comprising:
acquiring an endoscopic image photographed by an endoscope;
partitioning the acquired endoscopic image into a plurality of regions;
generating depth information indicating a depth of the acquired endoscopic image; and
specifying a direction in which the endoscope is advanceable based on region information indicating a result of segmentation and the depth information of the endoscopic image.

### Claim 9

A computer program storage medium that stores programs, wherein the programs cause a computer to implement:
a function of acquiring an endoscopic image photographed by an endoscope;
a function of partitioning the acquired endoscopic image into a plurality of regions;
a function of generating depth information indicating a depth of the acquired endoscopic image; and
a function of specifying a direction in which the endoscope is advanceable based on region information indicating a result of segmentation and the depth information of the endoscopic image.
