# US 2016/0354166 A1 · Robotic control of surgical instrument visibility

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0354166 A1
- Document ID: US20160354166A1
- Status: Published application
- Kind: A1
- Application number: 15/117,001
- Filed: 2015-01-29
- Published: 2016-12-08
- Earliest claimed: 2014-02-12
- Assignees: Koninklijke Philips N.V.
- Applicants: Koninklijke Philips N.V.
- Inventors: ALEKSANDRA POPOVIC; DAVID PAUL NOONAN
- Web page: https://patentgrep.com/patent/US20160354166A1

## CPC

- A61B 34/30: Surgical robots
- A61B 1/00006: {of control signals}
- A61B 1/000094: {extracting biological structures}
- A61B 1/00045: {Display arrangement}
- A61B 1/00133: {Drive units for endoscopic tools inserted through or with the endoscope}
- A61B 1/00149: {using articulated arms}
- A61B 1/045: Control thereof
- A61B 1/3132: {for laparoscopy}
- A61B 17/00234: {for minimally invasive surgery (A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence)}
- A61B 2034/2057: {Details of tracking cameras}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- A61B 2034/301: {for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

A robot guiding system employing a robot unit (10) including an endoscope (12) and a robot (11), and a control unit (20) including an endoscopic image controller (22) and a robot controller (21). In operation, the endoscope (12) generate an endoscopic image of an anatomical region as the robot (11) move the endoscope (12) within the anatomical region in response to robot actuator commands. The endoscopic image controller (22) controls a display of the endoscopic image (14) of the anatomical region and generates endoscope pose commands to maintain a visibility of two or more interventional instruments within the display of the endoscopic image (14) relative to a center of the endoscopic image (14). The robot controller (21) generates the robotic actuator commands responsive to the endoscope pose commands.

## Claims

### Claim 1

A robot guiding system, comprising:
a robot unit including
an endoscope operable to generate an endoscopic image of an anatomical region, and
a robot operably connected to the endoscope a to move the endoscope within the anatomical region responsive to robotic actuator commands; and
a control unit including
an endoscopic image controller operably connected to the endoscope to control a display of the endoscopic images of the anatomical region,
wherein the endoscopic image controller is operable to generate endoscope pose commands to maintain a visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image based on a selected spatial point positioned between the at least two interventional instruments,
a robot controller operably connected to the robot and the endoscopic image controller to generate the robotic actuator commands responsive to the endoscope pose commands.

### Claim 2 (depends on 1)

The robot guiding system of claim 1, wherein the endoscopic image controller aligns the spatial point between the at least two interventional instruments to the center of the endoscopic image.

### Claim 3 (depends on 2)

The robot guiding system of claim 2, wherein the spatial point is along at least one virtual line connecting the at least two interventional instruments.

### Claim 4 (depends on 3)

The robot guiding system of claim 3, wherein the spatial point is along a midpoint of the at least one virtual line.

### Claim 5 (depends on 3)

The robot guiding system of claim 3, wherein the spatial point is offset from a midpoint of the at least one virtual line.

### Claim 6 (depends on 2)

The robot guiding system of claim 2, wherein the endoscopic image controller generates the endoscope pose commands to pivot the endoscope as required to maintain an alignment of the spatial point between the at least two interventional instruments to the center of the endoscopic image.

### Claim 7 (depends on 2)

The robot guiding system of claim 2, wherein the endoscopic image controller generates the endoscope pose commands to longitudinally move the endoscope as required to maintain visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image responsive to the at least two interventional instruments being moved toward each other within the anatomical region.

### Claim 8 (depends on 2)

The robot guiding system of claim 2, wherein the endoscopic image controller generates the endoscope pose commands to longitudinally move the endoscope as required to maintain the visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image responsive to the at least two interventional instruments being moved away from other within the anatomical region.

### Claim 9 (depends on 2)

The robot guiding system of claim 2, wherein the endoscopic image controller generates the endoscope pose commands to rotate the endoscope as required to maintain the visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image responsive to the at least two interventional instruments being rotated relative to each other within the anatomical region.

### Claim 10 (depends on 1)

The robot guiding system of claim 1, wherein the endoscope is uncalibrated.

### Claim 11

A control unit for a robot connected to an endoscope generating an endoscopic image of an anatomical region, the control unit comprising:
an endoscopic image controller operable to control a display of the endoscopic images of the anatomical region,
wherein the endoscopic image controller is operable to generate endoscope pose commands to maintain a visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image based on a selected spatial point positioned between the at least two interventional instruments; and
a robot controller operably connected to the endoscopic image controller to generate robotic actuator commands responsive to the endoscope pose commands.

### Claim 12 (depends on 11)

The control unit of claim 11, wherein the endoscopic image controller aligns the spatial point between the at least two interventional instruments to the center of the endoscopic image.

### Claim 13 (depends on 12)

The control unit of claim 12, wherein the endoscopic image controller generates the endoscope pose commands to pivot the endoscope as required to maintain an alignment of the spatial point between the at least two interventional instruments to the center of the endoscopic image.

### Claim 14 (depends on 12)

The control unit of claim 12,
wherein the endoscopic image controller generates the endoscope pose commands to longitudinally move the endoscope as required to maintain visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image responsive to the at least two interventional instruments being moved toward each other within the anatomical region; and
wherein the endoscopic image controller generates the endoscope pose commands to longitudinally move the endoscope as required to maintain visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image responsive to the at least two interventional instruments being moved away from each other within the anatomical region.

### Claim 15 (depends on 12)

The control unit of claim 12, wherein the endoscopic image controller generates the endoscope pose commands to rotate the endoscope as required to maintain the visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image responsive to the at least two interventional instruments being rotated relative to each other within the anatomical region.

### Claim 16

A robot guiding method, comprising:
operating an endoscope to generate and display an endoscopic image of an anatomical region; and
commanding a robot to move the endoscope within the anatomical region as the endoscope is generating the endoscopic image of the anatomical region,
wherein the commanding of the robot maintains a visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image based on a selected spatial point positioned between the at least two interventional instruments.

### Claim 17 (depends on 16)

The robot guiding method of claim 16, wherein the spatial point between the at least two interventional instruments is aligned to the center of the endoscopic image.

### Claim 18 (depends on 17)

The robot guiding method of claim 17, wherein the robot is commanded to pivot the endoscope as required to maintain an alignment of the spatial point between the at least two interventional instruments to the center of the endoscopic image.

### Claim 19 (depends on 17)

The robot guiding method of claim 17,
wherein the robot is commanded to longitudinally move the endoscope as required to maintain visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image responsive to the at least two interventional instruments being moved toward each other within the anatomical region; and
wherein the robot is commanded to longitudinally move the endoscope as required to maintain visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image responsive to the at least two interventional instruments being moved away from other within the anatomical region.

### Claim 20 (depends on 17)

The robot guiding method of claim 17, wherein the robot is commanded to rotate the endoscope as required to maintain the visibility of at least two interventional instruments within the display of the endoscopic image relative to a center of the endoscopic image responsive to the at least two interventional instruments being rotated relative to each other within the anatomical region.
