# US 9,516,993 B2 · Endoscope system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,516,993 B2
- Document ID: US9516993B2
- Status: Granted
- Kind: B2
- Application number: 14/736,710
- Filed: 2015-06-11
- Published: 2015-10-01
- Granted: 2016-12-13
- Earliest claimed: 2013-03-27
- Assignees: Olympus Corporation
- Applicants: Olympus Corporation
- Inventors: Mitsuhiro Ito; Syunya Akimoto; Junichi Onishi
- Web page: https://patentgrep.com/patent/US9516993B2

## CPC

- A61B 1/000094: {extracting biological structures}
- A61B 1/000095: {for image enhancement}
- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- A61B 1/00055: {for alerting the user}
- A61B 1/00193: {adapted for stereoscopic vision}
- A61B 1/00194: {adapted for three-dimensional imaging}
- A61B 1/2676: {Bronchoscopes}
- A61B 5/0084: {for introduction into the body, e.g. by catheters (A61B5/1459 takes precedence)}
- A61B 5/066: {Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging}
- A61B 5/1076: {for measuring dimensions inside body cavities, e.g. using catheters (A61B3/1005 takes precedence)}
- A61B 5/1079: {using optical or photographic means}
- A61B 5/7246: {using correlation, e.g. template matching or determination of similarity}
- A61B 5/7282: {Event detection, e.g. detecting unique waveforms indicative of a medical condition (cough events A61B5/0823; seizures A61B5/4094; sleep apnoea A61B5/4818)}
- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 6/12: Arrangements for detecting or locating foreign bodies
- A61B 6/5223: {generating planar views from image data, e.g. extracting a coronal view from a 3D image}
- A61B 6/5247: {combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound}
- G06T 19/003: {Navigation within 3D models or images}
- A61B 1/0002: {provided with data storages}
- A61B 5/065: {Determining position of the probe employing exclusively positioning means located on or in the probe, e.g. using position sensors arranged on the probe}
- A61B 5/7207: {of noise induced by motion artifacts}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- G06T 2207/10068: Endoscopic image
- G06T 2210/41: Medical
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An endoscope system includes an endoscope including an image pickup section that picks up an image of a subject lumen organ, a recording section that records three-dimensional image information of a lumen organ acquired from a medical diagnosis apparatus, a position estimating section that estimates a distal end position of the endoscope, a lumen-diameter acquiring section that acquires a lumen diameter of the organ in the estimated position of the distal end, a condition determining section that determines whether the acquired lumen diameter is smaller than the reference lumen diameter, an image-change-amount detecting section that detects a change amount of a predetermined parameter in an endoscopic image picked up by the image pickup section, a virtual-endoscopic-image generating section that generates a virtual endoscopic image from any visual point position, and an information recording section that records a position of a distal end of an insertion section and the virtual endoscopic image.

## Claims

### Claim 1

An endoscope system comprising:
an endoscope including an image sensor configured to pick up an image of a lumen organ of a subject;
a first memory configured to record three-dimensional image information of a lumen organ of the subject acquired from a medical diagnosis apparatus different from the endoscope; and
a processor comprising hardware, wherein the processor is configured to:
estimate a position of a distal end of the endoscope;
acquire, on the basis of the three-dimensional image information recorded by the first memory, a lumen diameter of the lumen organ of the subject at the position of the distal end of the endoscope estimated;
determine, on the basis of the lumen diameter acquired and a reference lumen diameter set in advance, whether the lumen diameter acquired is smaller than the reference lumen diameter;
detect, when the processor determines that the lumen diameter acquired is smaller than the reference lumen diameter, a change amount of a predetermined parameter concerning an endoscopic image picked up by the image sensor involved in an inserting action of the endoscope, from the endoscope image; and
generate, on the basis of the three-dimensional image information recorded by the first memory, a virtual endoscopic image endoscopically rendered from any visual point position,
wherein the endoscope system further comprises:
a second memory configured to record, when the change amount of the predetermined parameter concerning the endoscopic image detected is larger than a reference set value set in advance, the position of the distal end of the endoscope and the virtual endoscopic image corresponding to the position of the distal end of the endoscope.

### Claim 2 (depends on 1)

The endoscope system according to claim 1,
wherein the processor is further configured to:
determine whether the change amount of the predetermined parameter detected changes by the reference set value set in advance or more, and
when the processor determines that the change amount of the predetermined parameter changes by the reference set value set in advance or more, the second memory is configured to record the virtual endoscopic image corresponding to the position of the distal end of the endoscope.

### Claim 3 (depends on 2)

The endoscope system according to claim 2,
wherein, when it is determined that the change amount of the predetermined parameter changes by the reference set value set in advance or more, the second memory is configured to further record information concerning an axial direction of the distal end of the endoscope.

### Claim 4 (depends on 1)

The endoscope system according to claim 1,
wherein the processor is configured to acquire a lumen diameter of the lumen organ on a surface perpendicular to a longitudinal direction of the distal end of the endoscope.

### Claim 5 (depends on 1)

The endoscope system according to claim 1,
wherein the processor is configured to acquire a lumen diameter of the lumen organ on a surface perpendicular to a core line, which is a center line of a lumen of the predetermined lumen organ.

### Claim 6 (depends on 1)

The endoscope system according to claim 1,
wherein the predetermined parameter is a shape in a branch region where a lumen of the lumen organ branches, and
wherein the processor is configured to detect a change amount of the shape in the branch region where the lumen of the lumen organ branches in the endoscopic image.

### Claim 7 (depends on 6)

The endoscope system according to claim 6,
wherein the processor is configured to detect whether length of a branch boundary where the lumen branches or a direction in a longitudinal direction of the branch boundary changes by a set value or more as the shape change amount in the branch region where the lumen of the lumen organ branches.

### Claim 8 (depends on 1)

The endoscope system according to claim 1,
wherein the predetermined parameter is brightness information in a branch region where a lumen of the lumen organ branches, and
wherein the processor is configured to detect the change amount of the brightness information in the branch region where the lumen of the lumen organ branches.

### Claim 9 (depends on 1)

The endoscope system according to claim 1,
wherein the processor is configured to:
compare the endoscopic image picked up by the image sensor and the virtual endoscopic image generated; and
estimate the position of the distal end of the endoscope on the basis of a comparison result of the endoscope image and the virtual endoscopic image.

### Claim 10 (depends on 9)

The endoscope system according to claim 9,
wherein the processor is configured to:
generate a lumen shape image of the predetermined lumen organ; and
perform control to, when the processor fails in an estimation of the position of the distal end of the endoscope based on a comparison result of the endoscope image and the virtual endoscopic image or when an instruction signal for presenting at least one information of the position of the distal end of the endoscope and the virtual endoscopic image corresponding to the position of the distal end of the endoscope is generated, display the virtual endoscopic image recorded in the second memory in a corresponding position in the lumen shape image, and
wherein the processor is configured to acquire the position of the distal end of the endoscope by comparing the virtual endoscopic image read out from the second memory and a present endoscopic image picked up by the image sensor.

### Claim 11 (depends on 1)

The endoscope system according to claim 1,
wherein, when the change amount detected by the processor continuously changes in a state of exceeding the reference set value, the second memory is configured to record the virtual endoscopic image when the change amount is a maximum.

### Claim 12 (depends on 1)

The endoscope system according to claim 1, further comprising:
a third memory configured to record a plurality of pieces of condition information as each of the reference set values set in advance concerning the reference lumen diameter and the predetermined parameter; and
an input device configured to selectively designate, from the third memory, condition information respectively used as the reference set values set in advance concerning the reference lumen diameter and the predetermined parameter.

### Claim 13 (depends on 1)

The endoscope system according to claim 1,
wherein the processor is configured to estimate a posture of the distal end of the endoscope in addition to the position of the distal end of the endoscope, and
wherein when the change amount of the predetermined parameter concerning the endoscopic image detected is larger than the reference set value set in advance, the second memory is configured to record the position and the posture of the distal end of the endoscope.