# US 11,992,349 B2 · System and method for local three dimensional volume reconstruction using a standard fluoroscope > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,992,349 B2 - Document ID: US11992349B2 - Status: Granted - Kind: B2 - Application number: 18/101,032 - Filed: 2023-01-24 - Published: 2023-05-25 - Granted: 2024-05-28 - Earliest claimed: 2015-08-06 - Assignees: Covidien LP - Applicants: Covidien LP - Inventors: Ron Barak; Oren P. Weingarten; Ariel Birenbaum; Guy Alexandroni - Web page: https://patentgrep.com/patent/US11992349B2 ## CPC - A61B 6/12: Arrangements for detecting or locating foreign bodies - A61B 6/00: Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment (instruments measuring radiation intensity for application in the field of nuclear medicine, e.g. in vivo counting, G01T1/161) - A61B 6/022: {Stereoscopic imaging} - A61B 6/463: {characterised by displaying multiple images or images and diagnostic data on one display} - A61B 6/466: {adapted to display 3D data} - A61B 6/487: {involving fluoroscopy} - A61B 6/5235: {combining images from the same or different ionising radiation imaging techniques, e.g. PET and CT} - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 6/0492: {using markers or indicia for aiding patient positioning} - A61B 6/4441: {the rigid structure being a C-arm or U-arm} - A61B 6/582: {Calibration} - A61B 2034/2046: {Tracking techniques} - A61B 2034/2051: {Electromagnetic tracking systems} - A61B 2034/2065: {Tracking using image or pattern recognition} - A61B 2090/367: {creating a 3D dataset from 2D images using position information} - A61B 2090/3762: {using computed tomography systems [CT]} - A61B 2090/3908: {Soft tissue, e.g. breast tissue} - A61B 2090/3966: {Radiopaque markers visible in an X-ray image} - A61B 2090/3983: {Reference marker arrangements for use with image guided surgery} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract A system for constructing fluoroscopic-based three-dimensional volumetric data of a target area within a patient from two-dimensional fluoroscopic images including a structure of markers, a fluoroscopic imaging device configured to acquire a sequence of images of the target area and of the structure of markers, and a computing device. The computing device is configured to estimate a pose of the fluoroscopic imaging device for at least a plurality of images of the sequence of images based on detection of a possible and most probable projection of the structure of markers as a whole on each image of the plurality of images. The computing device is further configured to construct fluoroscopic-based three-dimensional volumetric data of the target area based on the estimated poses of the fluoroscopic imaging device. ## Claims ### Claim 1 A system for constructing a three dimensional volume, comprising: a robotic surgical system for driving a medical device within a patient; a tracking system for determining a location of a portion of the robotically driven medical device with the patient; and a computing device including a processor and a display configured to display a graphical user interface, and a computer readable storage medium storing thereon instructions that, when executed by the processor cause the processor to: receive a sequence of intra-procedural images from an imaging device, each image comprising a portion of the robotically driven medical device; construct a three-dimensional volume based on the sequence of intra-procedural images, the three-dimensional volume including the portion of the robotically driven medical device and a target; determine a position of a target in the three-dimensional volume; determine a position of a distal portion of the robotically driven medical device in the received sequence of intra-procedural images; and determine an offset between the determined position of the distal portion of the robotically driven medical device and the determined position of the target. ### Claim 2 (depends on 1) The system of claim 1, wherein the instructions, when executed by the processor, determine a pose of the imaging device at which each of the received sequence of intra-procedural images was acquired based on detection of a projection of the medical device on the sequence of images. ### Claim 3 (depends on 1) The system of claim 1, wherein the instructions, when executed by the processor, determine a pose of the imaging device at which each of the received sequence of intra-procedural images was acquired using a structure-from-motion technique. ### Claim 4 (depends on 1) The system of claim 1, wherein the instructions, when executed by the processor, determine a pose of the imaging device at which each of the received sequence of intra-procedural images was acquired using an angle measurement device. ### Claim 5 (depends on 1) The system of claim 1, wherein the instructions, when executed by the processor, determine a pose of the imaging device at which each of the received sequence of intra-procedural images was acquired by registering a plurality of virtual images from a pre-procedure image data set to the received sequence of images. ### Claim 6 (depends on 1) The system of claim 1, wherein the instructions, when executed by the processor, present the three-dimensional volume constructed from the sequence of images in the user interface on the display. ### Claim 7 (depends on 1) The system of claim 1, wherein the instructions, when executed by the processor, present a re-projected two-dimensional image from the three-dimensional volume in the user interface on the display. ### Claim 8 (depends on 1) The system of claim 1, wherein the instructions, when executed by the processor, utilize the determined position of the medical device and the determined position of the target to correct a displayed position of the medical device in a navigation system. ### Claim 9 (depends on 1) The system of claim 1, wherein the received sequence of intra-procedural images is acquired from a fluoroscopic sweep of at least about 120 degrees. ### Claim 10 (depends on 1) The system of claim 1, wherein the medical device is a microwave ablation catheter. ### Claim 11 A system for constructing a three-dimensional volume, comprising: a robotic surgical system for driving a medical device within a patient; a tracking system for determining a location of a portion of the robotically driven medical device with the patient; and a computing device including a processor and a display configured to display a graphical user interface, and a computer readable storage medium storing thereon instructions that when executed by the processor cause the processor to: receive a sequence of intra-procedural images from an imaging device, each image comprising at least a portion of a structure of markers; construct a three-dimensional volume of a target area based on the sequence of intra-procedural images, the three-dimensional volume including a portion of the robotically driven medical device and a target; determine a position of the target in the three-dimensional volume; determine a position of a distal portion of the robotically driven medical device in a plurality of the intra-procedural images; and determine an offset between the determined position of the distal portion of the robotically driven medical device and the determined position of the target. ### Claim 12 (depends on 11) The system of claim 11, wherein the instructions, when executed by the processor, determine a pose of an intra-procedural imaging device based on detection of a projection of the markers on the sequence of intra-procedural images. ### Claim 13 (depends on 11) The system of claim 11, wherein the instructions, when executed by the processor, determine a pose of the intra-procedural imaging device at which each of the sequence of intra-procedural images was acquired using a structure-from-motion technique. ### Claim 14 (depends on 11) The system of claim 11, wherein the instructions when executed by the processor, determine a pose of the imaging device at which each of the sequence of intra-procedural images was acquired using an angle measurement device. ### Claim 15 (depends on 11) The system of claim 11, wherein the instructions, when executed by the processor, determine a pose of the imaging device at which each of the sequence of intra-procedural images was acquired by registering a plurality of virtual images from a pre-procedure image data set to the received sequence of intra-procedural images. ### Claim 16 (depends on 11) The system of claim 11, wherein the instructions, when executed by the processor, present the three-dimensional volume constructed from the sequence of intra-procedural images in the user interface on the display. ### Claim 17 (depends on 11) The system of claim 11, wherein the instructions, when executed by the processor, present a re-projected two-dimensional image from the three-dimensional volume in the user interface on the display. ### Claim 18 (depends on 11) The system of claim 11, wherein the instructions, when executed by the processor, utilize the determined position of the distal portion of the robotically driven medical device and the determined position of the target to correct a displayed position of the distal portion of the robotically driven medical device in a navigation system. ### Claim 19 (depends on 11) The system of claim 11, wherein the medical device is a microwave ablation catheter. ### Claim 20 A system for constructing a three dimensional volume, comprising: a robotic surgical system for driving a medical device within a patient; a tracking system for determining a location of a portion of the robotically driven medical device with the patient; and a computing device including a processor and a display configured to display a graphical user interface, and a computer readable storage medium storing thereon instructions that when executed by the processor cause the processor to: receive a sequence of images, each image comprising at least a portion of the robotically driven medical device; estimate a pose of an imaging device for each image of the sequence of images based on detection of a projection of a distal portion of the robotically driven medical device in the sequence of images; construct a three-dimensional volume based on the estimated poses of the imaging device for each image of the sequence of images, the three-dimensional volume including the distal portion of the robotically driven medical device and a target; determine a position of the target in the three-dimensional volume; determine a position of the distal portion of the robotically driven medical device in a plurality of the received sequence of images; and determine an offset between the position of the distal portion medical device and the target.