# US 10,765,484 B2 · Systems and methods for navigation and visualization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,765,484 B2
- Document ID: US10765484B2
- Status: Granted
- Kind: B2
- Application number: 16/113,666
- Filed: 2018-08-27
- Published: 2019-08-22
- Granted: 2020-09-08
- Earliest claimed: 2015-10-21
- Assignees: P Tech, LLC
- Applicants: P Tech, LLC
- Inventors: Peter M. Bonutti; Justin E. Beyers
- Web page: https://patentgrep.com/patent/US10765484B2

## CPC

- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B 34/30: Surgical robots
- A61B 90/361: {Image-producing devices, e.g. surgical cameras}
- A61B 90/37: {Surgical systems with images on a monitor during operation}
- G06T 7/11: Region-based segmentation
- G06T 7/194: involving foreground-background segmentation
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- A61B 90/98: using electromagnetic means, e.g. transponders
- A61B 2034/2048: {using an accelerometer or inertia sensor}
- A61B 2034/2055: {Optical tracking systems}
- A61B 2034/2057: {Details of tracking cameras}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image}
- A61B 2090/376: {using X-rays, e.g. fluoroscopy}
- A61B 2090/378: {using ultrasound}
- A61B 2090/392: {Radioactive markers}
- A61B 2090/3941: {Photoluminescent markers}
- G06T 2207/30008: Bone
- G06T 2207/30012: Spine; Backbone
- G06T 2207/30016: Brain
- G06T 2210/41: Medical
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system for visualizing a surgical site is provided. The system includes a robotic mechanism for performing a procedure on a patient, an imaging device coupled to the robotic mechanism, the imaging device configured to provide image data of a site of interest, and a computing device coupled to the imaging device. The computing device includes one or more processors and at least one memory device configured to store executable instructions. The executable instructions, when executed by the processor, are configured to receive the image data of the site of interest, track motion patterns of the site of interest in the received image data, filter the received image data to remove line-of-sight restrictions therein and alter pixels therein based on the tracked motion patterns, and generate an output frame from the filtered image data. The system also includes a presentation interface device coupled to the computing device and configured to present the output frame for visualization of the site of interest.

## Claims

### Claim 1

A system for visualizing a surgical site, the system comprising:
a robotic mechanism for performing a procedure on a patient;
an imaging device coupled to the robotic mechanism, the imaging device configured to provide image data of a site of interest;
a computing device coupled to the imaging device, the computing device including one or more processors and at least one memory device configured to store executable instructions, the executable instructions, when executed by the processor, configured to:
receive the image data of the site of interest;
differentiate between at least one object of interest and at least one detected obstruction;
filter the received image data to remove the at least one detected obstruction and recreate pixels related to the at least one detected obstruction based on at least one of (i) one or more buffered frames and (ii) one or more pixels adjacent to the pixels related to the at least one detected obstruction;
generate an output frame from the filtered image data; and
a presentation interface device coupled to the computing device, the presentation interface device configured to present the output frame for visualization of the site of interest.

### Claim 2 (depends on 1)

The system of claim 1, wherein pixels related to the at least one detected obstruction are recreated based on both (i) one or more buffered frames and (ii) one or more pixels adjacent to the at least one detected obstruction.

### Claim 3 (depends on 1)

The system of claim 1, wherein the at least one detected obstruction is at least one of blood, synovium, tissue fats, debris, bone fragments, and smoke.

### Claim 4 (depends on 1)

The system of claim 1, wherein pixels related to the at least one detected obstruction are recreated based on (i) one or more buffered frames.

### Claim 5 (depends on 4)

The system of claim 4, wherein the one or more buffered frames comprise a baseline image.

### Claim 6 (depends on 5)

The system of claim 5, wherein the computing system is configured to compare the received image data to the baseline image.

### Claim 7 (depends on 6)

The system of claim 6, wherein the computing system is configured to determine based on the comparison of the received image data to the baseline image that the detected obstruction is obstructing the object of interest in the received image data.

### Claim 8 (depends on 5)

The system of claim 5, wherein the computing system is configured to mask the received image data to superimpose pixels related to the object of interest based on the baseline image onto the received image data over the pixels related to the detected obstruction.

### Claim 9 (depends on 1)

The system of claim 1, wherein pixels related to the at least one detected obstruction are recreated based on (ii) pixels surrounding the at least one detected obstruction.

### Claim 10 (depends on 1)

The system of claim 1, wherein the computing device is configured to cease filtering when the at least one detected obstruction exceeds the predetermined size threshold.

### Claim 11 (depends on 1)

The system of claim 1, wherein the computing device is configured to alert when the at least one detected obstruction exceeds the predetermined size threshold.

### Claim 12 (depends on 1)

The system of claim 1, wherein the object of interest comprises a source of a bleed and the detected obstruction comprises an opaque blood flow.

### Claim 13 (depends on 1)

The system of claim 1, wherein the detected obstruction comprises opaque blood.

### Claim 14 (depends on 13)

The system of claim 13, wherein the computing device is configured to adjust a color value of the pixels related to the opaque blood based on a color value of one or more adjacent pixels.

### Claim 15 (depends on 13)

The system of claim 13, wherein the computing device is configured to adjust an intensity of the pixels related to the opaque blood based on an intensity of one or more adjacent pixels.

### Claim 16 (depends on 1)

The system of claim 1, wherein the computing device is configured to adjust a color value of the pixels related to the detected obstruction based on color values of one or more adjacent pixels.

### Claim 17 (depends on 1)

The system of claim 1, wherein computing device is configured to adjust the intensity of the pixels related to the detected obstruction based on color values of one or more adjacent pixels.