# US 2016/0353968 A1 · Endoscope system and method for operating endoscope system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0353968 A1
- Document ID: US20160353968A1
- Status: Published application
- Kind: A1
- Application number: 15/244,270
- Filed: 2016-08-23
- Published: 2016-12-08
- Earliest claimed: 2014-06-10
- Assignees: Olympus Corporation
- Applicants: Olympus Corporation
- Inventors: Soichi IKUMA; Tatsuro YAMAMOTO
- Web page: https://patentgrep.com/patent/US20160353968A1

## CPC

- A61B 1/00009: {of image signals during a use of endoscope}
- A61B 1/00: Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes (examination of body cavities or body tracts using ultrasonic, sonic or infrasonic waves A61B8/12; endoscopic instruments for taking cell samples or for biopsy A61B10/04); Illuminating arrangements therefor (for the eyes A61B3/00)
- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- A61B 1/00194: {adapted for three-dimensional imaging}
- A61B 5/06: Devices, other than using radiation, for detecting or locating foreign bodies {; Determining position of diagnostic devices within or on the body of the patient}
- G02B 23/24: Instruments {or systems} for viewing the inside of hollow bodies, e.g. fibrescopes
- A61B 1/044: {for absorption imaging}
- A61B 5/743: {Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots}
- A61B 2034/2051: {Electromagnetic tracking systems}
- A61B 2034/2061: {using shape-sensors, e.g. fiber shape sensors with Bragg gratings}
- A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

An endoscope system includes a portion configured to acquire a position of an objective optical system provided in an insertion portion of an endoscope; a portion configured to position the position of the objective optical system on three-dimensional image information about a subject; a portion configured to generate angle-of-view display information related to a picked-up image picked up by an image pickup system including the objective optical system; an instruction portion configured to output an instruction signal; and an image processing portion configured to create an overlapped image obtained by overlapping the angle-of-view display information at a position on the three-dimensional image information corresponding to the position of the objective optical system at a time point of inputting the instruction signal.

## Claims

### Claim 1

An endoscope system comprising:
an endoscope having an insertion portion to be inserted into a subject;
an objective optical system provided in the insertion portion and configured to form an optical image of an inside of the subject;
a position information acquiring portion configured to acquire position information about the objective optical system;
a positioning portion configured to position the position information acquired by the position information acquiring portion relative to three-dimensional image information related to the subject;
an angle-of-view display information generating portion configured to generate angle-of-view display information showing angle-of-view information about a picked-up image picked up by an image pickup system including the objective optical system;
an instruction portion configured to be operable by a user and output an instruction signal giving an instruction to overlap the angle-of-view display information generated by the angle-of-view display information generating portion on the three-dimensional image information; and
an image processing portion configured to create an overlapped image obtained by overlapping the angle-of-view display information at a position on the three-dimensional image information corresponding to the position information about the objective optical system acquired at a time point of inputting the instruction signal by the instruction portion.

### Claim 2 (depends on 1)

The endoscope system according to claim 1, wherein
the image processing portion continues to output the overlapped image generated at a time point of starting to input the instruction signal by the instruction portion for display while the instruction signal is effective.

### Claim 3 (depends on 1)

The endoscope system according to claim 1, further comprising
an accumulated display instruction portion configured to output an accumulated angle-of-view display instruction signal for accumulating and displaying past angle-of-view display information; wherein
when the accumulated angle-of-view display signal is inputted, the image processing portion creates an accumulated overlapped image obtained by accumulatively overlapping overlapped images obtained by overlapping the angle-of-view display information at a plurality of past time points on the three-dimensional image information.

### Claim 4 (depends on 1)

The endoscope system according to claim 1, wherein
the image processing portion is further capable of creating a two-dimensional overlapped image from the overlapped image; and
the endoscope system further comprises a second storage portion configured to store at least one of the two-dimensional overlapped image, and the position information and the angle-of-view information.

### Claim 5 (depends on 4)

The endoscope system according to claim 4, wherein the second storage portion stores at least one of the two-dimensional overlapped image, and the position information and the angle-of-view information in association with the picked-up image.

### Claim 6 (depends on 1)

The endoscope system according to claim 1, wherein the image processing portion creates the overlapped image in response to the position information being acquired by the position information acquiring portion.

### Claim 7 (depends on 1)

The endoscope system according to claim 1, further comprising an overlapped image selection instruction portion configured to receive a selection instruction for selecting on/off of overlapped image generation; wherein
the image processing portion generates the overlapped image when the overlapped image generation is turned on by the selection instruction and does not generate the overlapped image when the overlapped image generation is turned off by the selection instruction.

### Claim 8 (depends on 7)

The endoscope system according to claim 7, further comprising a display portion configured to display the overlapped image; wherein
the display portion displays the overlapped image while the overlapped image generation is turned on by the selection instruction.

### Claim 9 (depends on 1)

The endoscope system according to claim 1, wherein the image processing portion is further configured to create a plurality of two-dimensional overlapped images seen from a plurality of different directions, from the overlapped image.

### Claim 10 (depends on 1)

The endoscope system according to claim 1, wherein the angle-of-view display information includes boundary lines showing an angle-of-view range, the boundary lines extending with a position of the objective optical system shown by the position information as a starting point.

### Claim 11 (depends on 1)

The endoscope system according to claim 1, wherein, if a duration while the objective optical system is judged to be present in a predetermined area is equal to or above a first threshold based on the position information, the image processing portion automatically creates an overlapped image after the duration until a duration while the objective optical system is judged to be absent in the predetermined area is equal to or above a second threshold.

### Claim 12

A method for operating an endoscope system comprising an endoscope, an objective optical system, a position information acquiring portion, a positioning portion, an angle-of-view display information generating portion, an instruction portion and an image processing portion, the method comprising the steps of:
the objective optical system provided in an insertion portion of the endoscope forming an optical image of an inside of a subject;
the position information acquiring portion acquiring position information about the objective optical system;
the positioning portion positioning the position information acquired by the position information acquiring portion relative to three-dimensional image information related to the subject;
the angle-of-view display information generating portion generating angle-of-view display information showing angle-of-view information about a picked-up image picked up by an image pickup system including the objective optical system;
the instruction portion configured to be operable by a user outputting an instruction signal giving an instruction to overlap the angle-of-view display information generated by the angle-of-view display information generating portion on the three-dimensional image information; and
the image processing portion creating an overlapped image obtained by overlapping the angle-of-view display information at a position on the three-dimensional image information corresponding to the position information about the objective optical system acquired at a time point of inputting the instruction signal by the instruction portion.
