Systems and methods of visualizing a medical device relative to a target

A system for displaying a 3D rendering of a tool in a catheter relative to a lesion, comprising:

• a display;
• a processor in communication with the display;
• and
• a memory having stored thereon a neural network and instructions, which when executed by the processor, cause the processor to:
• receive first fluoroscopic images from a fluoroscopic imaging device centered on the catheter;
• estimate a pose of the fluoroscopic imaging device for each of the first fluoroscopic images to obtain estimated poses;
• generate a first three-dimensional (3D) reconstruction of the first fluoroscopic images using the estimated poses;
• apply a neural network to the first 3D reconstruction to determine a location and orientation of the tool in the first 3D reconstruction;
• receive a location of the lesion in the first 3D reconstruction;
• generate a 3D rendering of the tool relative to the lesion based on the location and orientation of the tool and the location of the lesion;
• and
• display the 3D rendering on the display.

A method for causing display of a 3D rendering of a tool in a catheter relative to a lesion, comprising:

• receiving first fluoroscopic images from a fluoroscopic imaging device centered on the catheter;
• estimating a pose of the fluoroscopic imaging device for each of the first fluoroscopic images to obtain estimated poses;
• generating a first three-dimensional (3D) reconstruction of the first fluoroscopic images utilizing the estimated poses;
• receiving two second fluoroscopic images at angles separated by at least 15 degrees;
• receiving locations of two points on the tool in the two second fluoroscopic images;
• receiving a location of the lesion;
• generating a 3D rendering of the tool relative to the lesion based on the locations of two points on the tool and the location of the lesion;
• and
• causing display of the 3D rendering.