Live 3D holographic guidance and navigation for performing interventional procedures

A method comprising:

• receiving, by a physical head-mounted device comprising a processor and a head-tracking mechanism, live tracking data from a procedure in 3D Cartesian coordinates of a navigation system,
• wherein the live tracking data comprises a position and orientation of:
• a tracked physical ultrasound transducer/probe connected to an ultrasound system,
• a tracked physical interventional device/instrument, and
• physical fiducial location sensors at specific anatomical locations on a physical patient;
• receiving, by the physical head-mounted device, a live ultrasound image stream acquired by the physical ultrasound transducer/probe connected to the ultrasound system;
• transforming, by the head-mounted device, the live ultrasound image stream to a headset coordinate system;
• transforming, by the head-mounted device, the live tracking data to the headset coordinate system;
• displaying, in a head-mounted display of the head mounted device, a live holographic projection of the live ultrasound image stream in the headset coordinate system, wherein the live holographic projection of the ultrasound image stream is based on the tracked position and tracked orientation of the transducer/probe, wherein the live holographic projection is scaled to the physical anatomy consistent with an operator's view assessed with the head-tracking mechanism;
• retrieving, by the head mounted device, digital anatomical objects derived from pre-operative 3D computed tomography (CT) image data of the physical patient, wherein the CT image data is in 3D coordinates of a CT coordinate system;
• transforming, by the head-mounted device, the digital anatomical objects from the 3D CT coordinate system to the headset coordinate system;
• translating, by the head mounted device, the anatomical objects in the headset coordinate system by a 3D vector computed based on a 3D point location on the live holographic projection of the live image stream and a corresponding point within the stored pre-operative CT image data in the headset coordinate system to correct for a live anatomical motion of the physical patient during the procedure;
• and
• displaying, in the head mounted display of the head mounted device, a holographic visualization comprising a holographic representation of the tracked physical interventional device/instrument congruent with the registered holographic projection of the live ultrasound image stream and the holographic anatomical objects derived from CT, wherein the holographic visualization is used to navigate the tracked physical ultrasound transducer/probe and guide the physical interventional device/instrument to a therapeutic target during the procedure.