Augmented reality viewing and tagging for medical procedures

A non-transitory machine readable storage medium having instructions embodied thereon, the instructions when executed cause a processor to augment medical imaging of a patient during a medical procedure using an augmented reality headset worn by a medical professional during the medical procedure, comprising:

• receiving a visual image of patient anatomy captured by a visual image camera during the medical procedure, the visual image comprising a viewable portion of the patient anatomy obtained during the medical procedure;
• identifying a patient marker in the visual image of the patient anatomy, the patient marker comprising information identifying the patient, information identifying patient anatomy that is the subject of the medical procedure, a patient orientation marker, or an image inversion prevention tag;
• retrieving an acquired medical image associated with the patient anatomy from a data store based in part on the patient marker, the acquired medical image comprising imaging acquired of one or more anatomical structures at a plurality of anatomical layers of the patient anatomy;
• anchoring the acquired medical image to the patient anatomy based in part on the patient orientation marker;
• retrieving an augmentation tag from data storage, the augmentation tag associated with the patient marker and a location in the acquired medical image, the augmentation tag identifying at least one anatomical structure of the acquired medical image found at the location, the augmentation tag further comprising a shape of the anatomical structure;
• and
• projecting, during the medical procedure, the acquired medical image and the augmentation tag onto lenses in an augmented reality headset to form a single graphical view which is overlaid on the patient anatomy viewable by the medical professional through the lenses of the augmented reality headset.

A system for augmenting a view of patent anatomy during a medical procedure for a medical professional using an augmented reality headset, comprising:

• a camera configured to obtain images of the patient anatomy during the medical procedure, the images comprising the view of the patient anatomy;
• an augmentation processor in communication with the camera and configured to:
• capture morphometric measurements of the patient anatomy from the images captured by the camera during the medical procedure;
• identify a patient marker in the images captured by the camera of the patient anatomy, the patient marker comprising information identifying the patient in order to retrieve pre-measured morphometric measurements;
• retrieving pre-measured morphometric measurements associated with the patient anatomy from data storage using the patient marker identified in the images captured by the camera;
• determine whether the morphometric measurements of the patent anatomy captured from the image match the pre-measured morphometric measurements associated with the patient anatomy as retrieved using the patient marker;
• retrieving an acquired medical image associated with the patient anatomy as defined by the patient marker and matched morphometric measurements from data storage, the acquired medical image comprising imaging acquired of one or more anatomical structures at a plurality of anatomical layers of the patient anatomy;
• aligning the acquired medical image with the view provided by the augmented reality headset during the medical procedure of the patient anatomy using the morphometric measurements;
• and
• form a single graphical view with the acquired medical image and an augmentation tag, the augmentation tag identifying at least one anatomical structure of the acquired medical image found at the location, the augmentation tag further comprising a shape of the anatomical structure;
• and
• an augmented reality headset and in communication with the augmentation processor and configured to:
• project the single graphical view formed from the acquired medical image and the augmentation tag onto lenses overlaid on the view of the patient anatomy during the medical procedure;
• and
• provide a notification the acquired medical image matches the patient anatomy.