# US 2024/0197410 A1 · Systems and methods for guiding drilled hole placement in endoscopic procedures

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0197410 A1
- Document ID: US20240197410A1
- Status: Published application
- Kind: A1
- Application number: 18/542,498
- Filed: 2023-12-15
- Published: 2024-06-20
- Earliest claimed: 2022-12-19
- Assignees: Stryker Corporation
- Applicants: Stryker Corporation
- Inventors: Brian FOUTS; Cole Kincaid HUNTER; Lina GUREVICH
- Web page: https://patentgrep.com/patent/US20240197410A1

## CPC

- A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B 1/000094: {extracting biological structures}
- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- A61B 34/25: {User interfaces for surgical systems}
- A61B 90/39: {Markers, e.g. radio-opaque or breast lesions markers}
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- A61B 1/000096: {using artificial intelligence}
- A61B 90/361: {Image-producing devices, e.g. surgical cameras}
- A61B 90/94: coded with symbols, e.g. text
- A61B 90/96: using barcodes
- A61B 2017/00203: {with speech control or speech recognition}
- A61B 2017/00207: {with hand gesture control or hand gesture recognition}
- A61B 2034/107: {Visualisation of planned trajectories or target regions}
- A61B 2034/2057: {Details of tracking cameras}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image}
- A61B 2090/3983: {Reference marker arrangements for use with image guided surgery}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for visually indicating a location of a hole to be drilled in tissue includes, at a computing system: receiving at least one endoscopic image that captures a surface of the tissue and at least a portion of a drill guide to be used for drilling the hole in the tissue; identifying in the at least one endoscopic image at least one fiducial marker disposed on the drill guide; determining a position and orientation of the at least one fiducial marker; determining a position and orientation of the hole to be drilled in the tissue based at least in part on the position and orientation of the at least one fiducial marker; and displaying a visualization of the tissue that comprises a graphical indication of the position and orientation of the hole to be drilled in the tissue.

## Claims

### Claim 1

A method for visually indicating a location of a hole to be drilled in tissue comprising, at a computing system:
receiving at least one endoscopic image that captures a surface of the tissue and at least a portion of a drill guide to be used for drilling the hole in the tissue;
identifying in the at least one endoscopic image at least one fiducial marker disposed on the drill guide;
determining a position and orientation of the at least one fiducial marker;
determining a position and orientation of the hole to be drilled in the tissue based at least in part on the position and orientation of the at least one fiducial marker; and
displaying a visualization of the tissue that comprises a graphical indication of the position and orientation of the hole to be drilled in the tissue.

### Claim 2 (depends on 1)

The method of claim 1, comprising: determining an orientation of the surface of the tissue, wherein the position and orientation of the hole to be drilled in the tissue is determined based on the orientation of the surface of the tissue.

### Claim 3 (depends on 2)

The method of claim 2, wherein the graphical indication comprises a graphical representation of an estimated break-through in the surface of the tissue.

### Claim 4 (depends on 3)

The method of claim 3, wherein a shape of the estimated break-through in the surface of the tissue is based on the orientation of the hole to be drilled and the orientation of the surface of the tissue.

### Claim 5 (depends on 3)

The method of claim 3, wherein the graphical representation of the estimated break-through in the surface of the tissue has a diameter that corresponds to a diameter of a drill bit.

### Claim 6 (depends on 5)

The method of claim 5, comprising receiving a user input corresponding to a selection of a different drill bit diameter and adjusting the diameter of the estimated break-through in the surface of the tissue based on the selection of the different drill bit diameter.

### Claim 7 (depends on 5)

The method of claim 5, wherein the visualization comprises a textual indication of the diameter of the drill bit.

### Claim 8 (depends on 1)

The method of claim 1, wherein the visualization comprises a graphical representation of a shaft of the hole to be drilled.

### Claim 9 (depends on 1)

The method of claim 1, wherein the surface of the tissue in the at least one endoscopic image is opposite a surface of the tissue where a drill bit will start drilling the hole to be drilled.

### Claim 10 (depends on 1)

The method of claim 1, wherein the visualization comprises an indication of a distance from the hole to be drilled to a landmark of the tissue.

### Claim 11 (depends on 10)

The method of claim 10, wherein the landmark of the tissue is at a user-selected location of the tissue.

### Claim 12 (depends on 10)

The method of claim 10, wherein the landmark of the tissue is automatically identified in the at least one endoscopic image.

### Claim 13 (depends on 10)

The method of claim 10, comprising receiving a user input selecting a different drill bit diameter and adjusting the indication of the distance from the hole to be drilled to the landmark of the tissue based on the different drill bit diameter.

### Claim 14 (depends on 2)

The method of claim 2, wherein determining the orientation of the surface of the tissue comprises:
generating a depth map that comprises depths at different locations of the surface of the tissue relative to a camera reference frame; and
determining normal vectors of the surface of the tissue based on the depth map.

### Claim 15 (depends on 2)

The method of claim 2, wherein determining the orientation of the surface of the tissue comprises:
determining relative motions of matched key points based on the at least one endoscopic image and a previously captured endoscopic image;
determining three-dimensional positions of key points within the at least one endoscope image based on the relative motions of matched key points; and
determining the orientation of the surface of the tissue based on the three-dimensional positions of key points associated with the surface of the tissue.

### Claim 16

A system comprising one or more processors, memory, and at least one program stored in the memory for execution by the one or more processors and comprising instructions that, when executed by the one or more processors, cause the system to:
receive at least one endoscopic image that captures a surface of the tissue and at least a portion of a drill guide to be used for drilling the hole in the tissue;
identify in the at least one endoscopic image at least one fiducial marker disposed on the drill guide;
determine a position and orientation of the at least one fiducial marker;
determine a position and orientation of the hole to be drilled in the tissue based on the position and orientation of the at least one fiducial marker; and
display a visualization of the tissue that comprises a graphical indication of the position and orientation of the hole to be drilled in the tissue.

### Claim 17

A method for visually indicating a location of a hole to be drilled in tissue comprising, at a computing system:
receiving at least one endoscopic image that captures a surface of the tissue and at least a portion of a tool;
identifying in the at least one endoscopic image and at least one fiducial marker disposed on the tool;
determining one or more characteristics of the at least one fiducial marker;
determining position information for the surface of the tissue based on the one or more characteristics of the at least one fiducial marker;
determining a position of at least one landmark of the tissue based on the position information for the surface of the tissue;
determining a position and orientation of the hole to be drilled in the tissue based on the position of the at least one landmark and a pre-operative plan; and
displaying a visualization of the tissue that comprises a graphical indication of the position and orientation of the hole to be drilled in the tissue.

### Claim 18 (depends on 17)

The method of claim 17, wherein determining position information for the surface of the tissue comprises generating a depth map that comprises depths at different locations of the surface of the tissue relative to a camera reference frame and calibrating the depth map using the one or more characteristics of the at least one fiducial marker.

### Claim 19 (depends on 17)

The method of claim 17, wherein determining the position of the at least one landmark comprises automatically identifying the at least one landmark.

### Claim 20 (depends on 17)

The method of claim 17, comprising, prior to determining the position of the at least one landmark, receiving a user input selecting the at least one landmark.
