# US 11,998,282 B2 · Systems and methods for teleoperated control of an imaging instrument

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,998,282 B2
- Document ID: US11998282B2
- Status: Granted
- Kind: B2
- Application number: 16/470,050
- Filed: 2017-12-15
- Published: 2020-01-16
- Granted: 2024-06-04
- Earliest claimed: 2016-12-16
- Assignees: Intuitive Surgical Operations, Inc.
- Applicants: Intuitive Surgical Operations, Inc.
- Inventors: Mahdi Azizian; Jonathan M. Sorger; Oliver J. Wagner; Andrew C. Waterbury
- Web page: https://patentgrep.com/patent/US11998282B2

## CPC

- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B 8/0841: {for locating instruments}
- A61B 17/3403: {Needle locating or guiding means}
- A61B 34/30: Surgical robots
- A61B 34/35: for telesurgery
- A61B 90/361: {Image-producing devices, e.g. surgical cameras}
- A61B 90/39: {Markers, e.g. radio-opaque or breast lesions markers}
- A61B 90/50: Supports for surgical instruments, e.g. articulated arms
- A61B 90/96: using barcodes
- A61B 1/00193: {adapted for stereoscopic vision}
- A61B 8/4416: {related to combined acquisition of different diagnostic modalities, e.g. combination of ultrasound and X-ray acquisitions}
- A61B 17/29: Forceps for use in minimally invasive surgery
- A61B 2010/045: {Needles (pointed biopsy devices with vacuum aspiration A61B10/0283)}
- A61B 2017/00477: {Coupling (A61B2017/0046 takes precedence)}
- A61B 2017/00876: {magnetic}
- A61B 2017/306: {holding by means of suction}
- A61B 2017/3407: {including a base for support on the body}
- A61B 2017/3413: {guided by ultrasound}
- A61B 2018/00577: {Ablation}
- A61B 2034/107: {Visualisation of planned trajectories or target regions}
- A61B 2034/2055: {Optical tracking systems}
- A61B 2034/2059: {Mechanical position encoders}
- A61B 2034/2061: {using shape-sensors, e.g. fiber shape sensors with Bragg gratings}
- A61B 2034/2063: {Acoustic tracking systems, e.g. using ultrasound}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- A61B 2034/2072: {Reference field transducer attached to an instrument or patient}
- A61B 2034/302: {specifically adapted for manipulations within body cavities, e.g. within abdominal or thoracic cavities}
- A61B 2090/364: {Correlation of different images or relation of image positions in respect to the body}
- A61B 2090/371: {with simultaneous use of two cameras}
- A61B 2090/3735: {Optical coherence tomography [OCT]}
- A61B 2090/3784: {both receiver and transmitter being in the instrument or receiver being also transmitter}
- A61B 2090/392: {Radioactive markers}
- A61B 2090/3937: {Visible markers}
- A61B 2090/3941: {Photoluminescent markers}
- A61B 2090/3945: {Active visible markers, e.g. light emitting diodes}
- A61B 2090/3966: {Radiopaque markers visible in an X-ray image}
- A61B 2090/3983: {Reference marker arrangements for use with image guided surgery}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

A medical imaging system comprises a teleoperated assembly including a medical instrument, a first imaging instrument, a second imaging instrument, a tracking fixture, and a processor. The first imaging instrument generates a first image of a field of view of the first imaging instrument and the second imaging instrument generates a second image. The tracking fixture is engageable along at least a portion of the second imaging instrument. The tracking fixture includes one or more features. The medical instrument is attachable to an attachment feature of the tracking fixture to maintain a known configuration of the medical instrument relative to the second imaging instrument. The processor is configured to determine a pose of the tracking fixture based on the one or more features of the tracking fixture and the first image.

## Claims

### Claim 1

A medical imaging system comprising:
a teleoperated assembly including a medical instrument;
a first imaging instrument for generating a first image of a field of view of the first imaging instrument;
a second imaging instrument for generating a second image; a tracking fixture engageable along at least a portion of the second imaging instrument, the tracking fixture including one or more features wherein the medical instrument is attachable to an attachment feature of the tracking fixture to maintain a known configuration of the medical instrument relative to the second imaging instrument; and
a processor configured to determine a pose of the tracking fixture based on the one or more features of the tracking fixture and the first image.

### Claim 2 (depends on 1)

The medical imaging system of claim 1 further comprising a control system of the teleoperated assembly for generating a modified second image based on information from the first image.

### Claim 3 (depends on 1)

The medical imaging system of claim 1 wherein the second imaging instrument includes a set of elongated grooves and the tracking fixture includes a set of projections sized for receipt within the set of elongated grooves to attach the tracking fixture to the second imaging instrument.

### Claim 4 (depends on 1)

The medical imaging system of claim 1 wherein the tracking fixture includes an expandable member.

### Claim 5 (depends on 1)

The medical imaging system of claim 1 wherein the one or more features are detectable in the first image.

### Claim 6 (depends on 1)

The medical imaging system of claim 1 wherein the first imaging instrument includes a camera and the second imaging instrument is an ultrasound probe.

### Claim 7 (depends on 1)

The medical imaging system of claim 1 wherein the second imaging instrument includes an optical fiber shape sensor for determining a pose of the second imaging instrument.

### Claim 8 (depends on 1)

The medical imaging system of claim 1 wherein a three- dimensional shape of the one or more features permits the pose of the tracking fixture to be determined based on the first image.

### Claim 9

A medical imaging system further comprising:
a teleoperated assembly including a medical instrument;
a first imaging instrument for generating a first image of a field of view of the first imaging instrument;
a second imaging instrument for generating a second image;
a tracking fixture engageable along at least a portion of the second imaging instrument, the tracking fixture including one or more features configured to permit a pose of the tracking fixture to be determined, wherein the medical instrument is attachable to the tracking fixture to maintain a known configuration of the medical instrument relative to the second imaging instrument;
a guiding member rigidly coupled to the tracking fixture, the guiding member including a channel sized for receipt of a medical tool while the tracking fixture is engaged along the portion of the second imaging instrument; and
a processor configured to determine a pose of the tracking fixture based on the one or more features of the tracking fixture appearing in the first image.

### Claim 10 (depends on 9)

The medical imaging system of claim 9 wherein a trajectory through the channel intersects an imaging area of the second image.

### Claim 11 (depends on 1)

The medical imaging system of claim 1 wherein the tracking fixture includes a coupling mechanism for attaching the medical instrument to the tracking fixture.

### Claim 12 (depends on 11)

The medical imaging system of claim 11 wherein the coupling mechanism includes a magnet.

### Claim 13 (depends on 12)

The medical imaging system of claim 12 wherein the coupling mechanism includes a magnet shield.

### Claim 14 (depends on 11)

The medical imaging system of claim 11 wherein the coupling mechanism includes a suction mechanism.

### Claim 15 (depends on 11)

The medical imaging system of claim 11 wherein the coupling mechanism includes a compliant joint mechanism.

### Claim 16 (depends on 15)

The medical imaging system of claim 15 further comprising a sensor associated with the compliant joint mechanism for determining a pose of the tracking fixture with respect to the medical instrument.

### Claim 17

A medical imaging system comprising:
a stereoscopic endoscopic instrument for generating a first image;
a supplemental imaging instrument for generating a second image;
a sleeve sized to extend along at least a portion of the supplemental imaging instrument, wherein the sleeve is visible within an imaging area of the first image, the sleeve including a marking scheme visible within the first image; and
a processor configured to:
receive the first image;
determine, based on the marking scheme visible within the first image, a pose of the sleeve relative to the stereoscopic endoscopic instrument; and
modify the second image based on the determined pose of the sleeve relative to the stereoscopic endoscopic instrument.

### Claim 18 (depends on 17)

The medical imaging system of claim 17 wherein the marking scheme includes a checkerboard pattern.

### Claim 19 (depends on 17)

The medical imaging system of claim 17 wherein the marking scheme includes at least one of a barcode pattern, a fluorescent marker, or a light emitting device.

### Claim 20 (depends on 17)

The medical imaging system of claim 17 wherein the processor is configured to modify the second image by generating a three-dimensional warped image in alignment with the first image.