# US 12,702,500 B2 · Controller, endoscope system, and control method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,702,500 B2
- Document ID: US12702500B2
- Status: Granted
- Kind: B2
- Application number: 18/105,305
- Filed: 2023-02-03
- Published: 2023-06-08
- Granted: 2026-08-11
- Earliest claimed: 2020-09-10
- Assignees: Olympus Corporation; National Cancer Center
- Applicants: Olympus Corporation; National Cancer Center
- Inventors: Hiroto Ogimoto; Ryota Sasai; Masaru Yanagihara; Hiro Hasegawa; Daichi Kitaguchi; Nobuyoshi Takeshita; Shigehiro Kojima; Yuki Furusawa; Yumi Kinebuchi; Masaaki Ito
- Web page: https://patentgrep.com/patent/US12702500B2

## CPC

- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- A61B 34/30: Surgical robots
- A61B 1/00006: {of control signals}
- A61B 1/00009: {of image signals during a use of endoscope}
- A61B 1/0004: {for electronic operation}
- A61B 1/00042: {for mechanical operation}
- A61B 1/00045: {Display arrangement}
- A61B 1/00149: {using articulated arms}
- A61B 1/0016: {using motor drive units}
- A61B 1/045: Control thereof
- A61B 5/065: {Determining position of the probe employing exclusively positioning means located on or in the probe, e.g. using position sensors arranged on the probe}
- A61B 34/32: operating autonomously
- G02B 23/24: Instruments {or systems} for viewing the inside of hollow bodies, e.g. fibrescopes
- G06T 3/60: Rotation of whole images or parts thereof
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- H04N 23/62: Control of parameters via user interfaces
- H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
- A61B 1/000094: {extracting biological structures}
- A61B 1/000096: {using artificial intelligence}
- A61B 1/00193: {adapted for stereoscopic vision}
- A61B 5/6847: {mounted on an invasive device}
- A61B 34/25: {User interfaces for surgical systems}
- A61B 2034/2055: {Optical tracking systems}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- A61B 2034/2068: {using pointers, e.g. pointers having reference marks for determining coordinates of body points}
- A61B 2034/301: {for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes}
- A61B 2034/305: {Details of wrist mechanisms at distal ends of robotic arms}
- A61B 2090/363: {Use of fiducial points}
- G06T 2207/10012: Stereo images
- G06T 2207/10068: Endoscopic image
- G06T 2207/30004: Biomedical image processing
- G06T 2207/30244: Camera pose
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A controller controls a movement of an endoscope to cause the endoscope to follow a surgical instrument. The controller includes a processor. The processor acquires position information including the position of the surgical instrument, acquires scene information associated with a procedure scene to be observed through the endoscope, determines an offset parameter, which determines the position of the target point with respect to a predetermined fiducial point in the field of view of the endoscope, of a target point on the basis of the scene information, sets the position of the target point with respect to the fiducial point on the basis of the offset parameter, and causes the endoscope to follow the surgical instrument such that the surgical instrument is disposed at the target point, by controlling a movement of the endoscope on the basis of the position of the target point and the position of the surgical instrument.

## Claims

### Claim 1

A controller comprising:
a processor comprising hardware, the processor being configured to:
acquire position information including a position of a surgical instrument,
acquire scene information related to a procedure of a surgical operation, the scene information being information associated with a procedure scene to be observed through an endoscope,
determine an offset parameter of a target point on a basis of the scene information, the offset parameter being a parameter that determines a position of the target point with respect to a predetermined fiducial point in a field of view of the endoscope,
set the position of the target point with respect to the fiducial point on a basis of the offset parameter, and
control a movement of the endoscope on a basis of the position of the target point and the position of the surgical instrument to three-dimensionally change a positional relationship between a tip of the surgical instrument and the endoscope such that the surgical instrument is disposed at the target point.

### Claim 2 (depends on 1)

The controller according to claim 1, wherein the processor is configured to:
acquire a moving vector of the surgical instrument as the scene information; and
determine the offset parameter on a basis of the moving vector.

### Claim 3 (depends on 2)

The controller according to claim 2, wherein the processor is configured to determine the offset parameter that causes a region ahead of the surgical instrument in a moving direction of the surgical instrument to lie at a center of the field of view, on a basis of a direction of the moving vector.

### Claim 4 (depends on 1)

The controller according to claim 1, wherein the processor is configured to estimate the procedure scene on the basis of the scene information and determines the offset parameter on a basis of the estimated procedure scene.

### Claim 5 (depends on 4)

The controller according to claim 4, wherein the processor is configured to acquire a type of the surgical instrument as the scene information and estimates the procedure scene on a basis of the type of the surgical instrument.

### Claim 6 (depends on 5)

The controller according to claim 5, wherein when the procedure scene is estimated to be an ablating scene, the processor is configured to determine the offset parameter that causes a region to be ablated by the surgical instrument to lie at a center of the field of view.

### Claim 7 (depends on 5)

The controller according to claim 5, wherein when the procedure scene is estimated to be an unfolding scene, the processor is configured to set the offset parameter that causes a region to be unfolded by the surgical instrument to lie at a center of the field of view.

### Claim 8 (depends on 4)

The controller according to claim 4, wherein the processor is configured to:
acquire an operating state of the surgical instrument as the scene information; and
estimate the procedure scene on a basis of the operating state of the surgical instrument.

### Claim 9 (depends on 4)

The controller according to claim 4, wherein the processor is configured to:
acquire anatomy information on a biological tissue in the field of view as the scene information; and
estimate the procedure scene on a basis of the anatomy information.

### Claim 10 (depends on 1)

The controller according to claim 1, wherein the processor is configured to acquire the position information including a three-dimensional position of the surgical instrument.

### Claim 11 (depends on 1)

An endoscope system comprising:
an endoscope;
a moving device that moves the endoscope in a subject; and
the controller according to claim 1.

### Claim 12 (depends on 11)

The endoscope system according to claim 11, wherein the endoscope captures an image in the subject and acquires the image in the subject.

### Claim 13 (depends on 11)

The endoscope system according to claim 11, wherein the moving device includes a robot arm that is connected to the endoscope and controls a position and orientation of the endoscope.

### Claim 14 (depends on 13)

The endoscope system according to claim 13, wherein the processor is configured to:
calculate the position and orientation of the endoscope such that the target point is disposed at a tip of the surgical instrument, and
control the robot arm on a basis of the calculated position and orientation of the endoscope.