# US 11,896,414 B2 · System and method for pose estimation of an imaging device and for determining the location of a medical device with respect to a target

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,896,414 B2
- Document ID: US11896414B2
- Status: Granted
- Kind: B2
- Application number: 17/845,936
- Filed: 2022-06-21
- Published: 2022-10-06
- Granted: 2024-02-13
- Earliest claimed: 2018-02-08
- Assignees: Covidien LP
- Applicants: Covidien LP
- Inventors: Ron Barak; Ariel Birenbaum; Guy Alexandroni; Oren P. Weingarten
- Web page: https://patentgrep.com/patent/US11896414B2

## CPC

- A61B 6/487: {involving fluoroscopy}
- G06T 7/0012: {Biomedical image inspection}
- A61B 1/00158: {using magnetic field}
- A61B 1/043: {for fluorescence imaging}
- A61B 5/064: {using markers (A61B5/062 takes precedence)}
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B 90/36: {Image-producing devices or illumination devices not otherwise provided for}
- A61B 90/39: {Markers, e.g. radio-opaque or breast lesions markers}
- G06T 7/248: {involving reference images or patches}
- G06T 7/251: {involving models}
- G06T 7/269: using gradient-based methods
- G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/73: using feature-based methods
- G06T 7/75: {involving models}
- G06T 7/77: using statistical methods
- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- A61B 1/042: {characterised by a proximal camera, e.g. a CCD camera}
- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 2034/2051: {Electromagnetic tracking systems}
- A61B 2034/2055: {Optical tracking systems}
- A61B 2090/364: {Correlation of different images or relation of image positions in respect to the body}
- A61B 2090/3983: {Reference marker arrangements for use with image guided surgery}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10068: Endoscopic image
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/10088: Magnetic resonance imaging [MRI]
- G06T 2207/10121: Fluoroscopy
- G06T 2207/30004: Biomedical image processing
- G06T 2207/30021: Catheter; Guide wire
- G06T 2207/30061: Lung
- G06T 2207/30096: Tumor; Lesion
- G06T 2207/30208: Marker matrix
- G06T 2207/30244: Camera pose
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system and method for constructing fluoroscopic-based three-dimensional volumetric data of a target area within a patient from two-dimensional fluoroscopic images acquired via a fluoroscopic imaging device.

## Claims

### Claim 1

A system for constructing three-dimensional volumetric data of a target area within a patient from two-dimensional (2D) images acquired via a rotatable imaging device, comprising:
a structure of markers, wherein a sequence of 2D images of the target area and of the structure of markers is acquired via a rotatable imaging device; and
a computing device configured to:
detect a most probable projection of the structure of markers as a whole on each image of at least a portion of the sequence of 2D images;
estimate a pose of the rotatable imaging device for the at least the portion of the sequence of 2D images based on the most probable projection of the structure of markers as a whole on each image of the at least the portion of the sequence 2D images, yielding estimated poses of the rotatable imaging device;
construct three-dimensional volumetric data of the target area from the at least the portion of the sequence 2D images based on the respective estimated poses of the rotatable imaging device;
determine a distance between a medical device and a target appearing in the three-dimensional volumetric data; and
correct a display of the medical device with respect to the target based on the distance.

### Claim 2 (depends on 1)

The system of claim 1, wherein the computing device is further configured to:
facilitate an approach of a medical device to the target area, wherein a medical device is positioned in the target area prior to acquiring the sequence of 2D images; and
determine an offset between the medical device and the target based on the three-dimensional volumetric data.

### Claim 3 (depends on 2)

The system of claim 2, further comprising a locating system indicating a location of the medical device within the patient, wherein the computing device comprises a display and is configured to:
display the target area and the location of the medical device with respect to the target;
facilitate navigation of the medical device to the target area via the locating system and the display; and
correct the display of the location of the medical device with respect to the target based on the offset between the medical device and the target.

### Claim 4 (depends on 3)

The system of claim 3, wherein the computing device is further configured to:
display a 3D rendering of the target area on the display; and
register the locating system to the 3D rendering, wherein correcting the display of the location of the medical device with respect to the target comprises updating a registration between the locating system and the 3D rendering.

### Claim 5 (depends on 3)

The system of claim 3, wherein the locating system is an electromagnetic locating system.

### Claim 6 (depends on 3)

The system of claim 3, wherein the target area comprises at least a portion of lungs and the medical device is navigable to the target area through airways of a luminal network.

### Claim 7 (depends on 1)

The system of claim 1, wherein the structure of markers is at least one of a periodic pattern or a two-dimensional pattern.

### Claim 8 (depends on 2)

The system of claim 2, wherein the target area comprises at least a portion of a lung and the target is a soft-tissue target.

### Claim 9

A method for constructing fluoroscopic-based three dimensional volumetric data of a target area within a patient from a sequence of two-dimensional (2D) images of a target area and of a structure of markers acquired via a rotatble imaging device, wherein the structure of markers is positioned between the patient and the rotatable imaging device, the method comprising using at least one hardware processor for:
detecting a most probable projection of the structure of markers as a whole on each image of at least a portion of the sequence of 2D images acquired via the rotatble imaging device;
estimating a pose of the rotatable imaging device for at least a portion of the sequence of 2D images based on the most probable projection of the structure of markers as a whole on each image of the at least the portion of the sequence of 2D images;
constructing three-dimensional volumetric data of the target area from the at least the portion of the 2D images based on the respective estimated poses of the rotatable imaging device;
determining a distance between a medical device and a target appearing in the three-dimensional volumetric data; and
correcting a display of the medical device with respect to the target based on the distance.

### Claim 10 (depends on 9)

The method of claim 9, wherein a medical device is positioned in the target area prior to acquiring the sequence of 2D images, and
wherein the method further comprises using the at least one hardware processor for determining an offset between the medical device and the target based on the three-dimensional volumetric data.

### Claim 11 (depends on 10)

The method of claim 10, further comprising using the at least one hardware processor for:
facilitating navigation of the medical device to the target area via a locating system indicating a location of the medical device and via a display; and
correcting a display of the location of the medical device with respect to the target based on the offset between the medical device and the target.

### Claim 12 (depends on 11)

The method of claim 11, further comprising using the at least one hardware processor for:
displaying a 3D rendering of the target area on the display; and
registering the locating system to the 3D rendering,
wherein the correcting of the location of the medical device with respect to the target comprises updating the registration of the locating system to the 3D rendering.

### Claim 13 (depends on 12)

The method of claim 12, further comprising using the at least one hardware processor for generating the 3D rendering of the target area based on previously acquired CT volumetric data of the target area.

### Claim 14 (depends on 10)

The method of claim 10, wherein the target area comprises at least a portion of lungs and wherein the medical device is navigable to the target area through airways of a luminal network.

### Claim 15 (depends on 11)

The method of claim 11, wherein the structure of markers is at least one of a periodic pattern or a two-dimensional pattern.

### Claim 16 (depends on 11)

The method of claim 11, wherein the target area comprises at least a portion of a lung and the target is a soft-tissue target.

### Claim 17

A system for constructing three-dimensional volumetric data of a target area within a patient from two-dimensional (2D) images acquired via a rotatable imaging device, comprising:
a computing device configured to:
detect a most probable projection of a structure of markers as a whole on each image of at least a portion of a sequence of 2D images acquired via a rotatable imaging device;
estimate a pose of the rotatable imaging device for the at least the portion of the sequence of 2D images based on the most probable projection of the structure of markers as a whole on each image of the at least the portion of the sequence of 2D images, yielding estimated poses of the rotatable imaging device;
construct three-dimensional volumetric data of the target area from the at least the portion of the sequence of 2D images based on the respective estimated poses of the rotatable imaging device;
determine a distance between a medical device and a target appearing in the three-dimensional volumetric data; and
correct a display of the medical device with respect to the target based on the distance.

### Claim 18 (depends on 17)

The system of claim 17, wherein the computing device is further configured to:
facilitate an approach of a medical device to the target area, wherein a medical device is positioned in the target area prior to acquisition of the sequence of 2D images; and
determine an offset between the medical device and the target based on the three-dimensional volumetric data.

### Claim 19 (depends on 18)

The system of claim 18, further comprising:
a locating system indicating a location of the medical device within the patient; and
a display,
wherein the computing device is further configured to:
display the target area and the location of the medical device with respect to the target on the display;
facilitate navigation of the medical device to the target area via the locating system and the display; and
correct the display of the location of the medical device with respect to the target based on the determined offset between the medical device and the target.

### Claim 20 (depends on 19)

The system of claim 19, wherein the computing device is further configured to:
display a 3D rendering of the target area on the display; and
register the locating system to the 3D rendering, wherein correcting the display of the location of the medical device with respect to the target comprises updating the registration between the locating system and the 3D rendering.