Mixed reality image guidance for medical interventions
AssigneeCornell University
InventorsBobak Mosadegh, Matin Torabinia
Life of this patent
Estimating the term…
Abstract
Disclosed are approaches that may provide image-guidance to interventionalists by providing true 3D visualization and quantitative feedback in real-time. A guidance system may allow a physician to manipulate a medical device and see a 3D rendering with quantitative feedback floating in mixed reality, next to standard monitors. Image tracking may detect and co-register the medical device's 3D position using, for example, bi-plane C-arm X-ray fluoroscopy and provide a 3D trajectory as quantitative feedback. Patterns in a fluoroscopic image may be used to accurately determine an object's z-position from a single angle projection.
Claims20
A method of providing real-time image guidance and/or navigation for a medical procedure, comprising:
- receiving a first image data set of an anatomical target in relation to at least one fiducial marker in a selected coordinate system;
- generating a 3D model of the anatomical target, in relation to the first image data set and the at least one fiducial marker, in the selected coordinate system;
- receiving at least one single-plane fluoroscopic image comprising the at least one fiducial marker and a medical device;
- generating a mask or a center coordinate of the at least one fiducial marker from the at least one single-plane fluoroscopic image;
- generating a 3D model of the medical device from the at least one single-plane fluoroscopic image in the selected coordinate system by performing feature extraction on the medical device and the at least one fiducial marker, and using a trained machine learning model to infer, based on extracted features, a z-position of at least one portion of the medical device;
- registering the 3D model of the medical device with the 3D model of the anatomical target in the selected coordinate system;
- and
- generating a real-time output image comprising the 3D model of the medical device registered with a geometry of the anatomical target in the selected coordinate system.
A system for image guidance comprising:
- one or more processors;
- and
- a non-transitory memory device storing processor executable instructions to cause the one or more processors, upon execution of the executable instructions, to:
- receive a first image data set of an anatomical target in relation to at least one fiducial marker in a selected coordinate system;
- generate a 3D model of the anatomical target, in relation to the first image data set and the at least one fiducial marker, in the selected coordinate system;
- receive at least one single-plane fluoroscopic image comprising the at least one fiducial marker and a medical device;
- generate a mask of the at least one fiducial marker from the at least one single-plane fluoroscopic image;
- generate a 3D model of the medical device from the at least one single-plane fluoroscopic image in the selected coordinate system by performing feature extraction on the medical device and the at least one fiducial marker, and using a trained deep learning model to infer, based on extracted features, a z-position of at least one portion of the medical device;
- register the 3D model of the medical device with the 3D model of the anatomical target in the selected coordinate system;
- and
- generate a real-time output image comprising the 3D model of the medical device registered with a geometry of the anatomical target in the selected coordinate system.
Dependencies are inferred from the claim text.