System and method for direct augmented reality medical registration

A method for multi-modal visualization and guidance in an augmented reality medical procedure, comprising:

• providing an augmented reality system and a tracking system with registration established between a tracking coordinate system, an augmented reality coordinate system, and an imaging coordinate system of pre-procedural medical imaging data;
• attaching a tracking sensor to an ultrasound probe;
• positioning the ultrasound probe on a patient to acquire a real-time ultrasound image;
• determining a position and orientation of an ultrasound image plane using the tracking sensor;
• displaying, using the augmented reality system, a holographic representation of the ultrasound image as a plane positioned in three-dimensional space at an actual location and orientation of the ultrasound image plane;
• fusing the real-time ultrasound image with the pre-procedural medical imaging data;
• and
• displaying guidance information for navigating a tracked interventional device to a target location.

A method for dynamically maintaining registration accuracy during an augmented reality medical procedure, comprising:

• establishing initial registration between a tracking coordinate system, an augmented reality coordinate system, and an imaging coordinate system;
• displaying holographic content based on the initial registration, the holographic content appearing spatially aligned with a patient's physical anatomy;
• monitoring for conditions affecting registration accuracy during the medical procedure;
• detecting occurrence of a condition affecting registration accuracy;
• determining an updated coordinate transformation that compensates for the detected condition;
• and
• applying the updated coordinate transformation to adjust positions of the displayed holographic content to maintain spatial alignment with the patient's physical anatomy.