Augmenting real-time views of a patient with three-dimensional data

A method for augmenting real-time views of a patient with three-dimensional (3D) data, the method comprising:

• identifying 3D data for a patient, the 3D data including an outer layer of the patient and multiple inner layers of the patient;
• mapping a 3D space;
• registering a real-time position of the outer layer of the patient in the 3D space by mapping the position of the outer layer of the patient within the 3D space;
• registering a virtual position of the outer layer of the patient from the 3D data to align with a real-time position of the outer layer of the patient in a 3D space;
• displaying, in an augmented reality (AR) headset, one of the inner layers of the patient from the 3D data projected onto real-time views of the outer layer of the patient, the real-time views being non-image actual views;
• determining real-time morphometric measurements, including size and shape, of an object prior to insertion of the object into the patient through the outer layer of the patient;
• automatically tracking a real-time position of the object in the 3D space with respect to the registered real-time position of the outer layer of the patient in the 3D space and with respect to the registered virtual position of the outer layer of the patient from the 3D data;
• and
• while a portion of the object is inserted into the patient through the outer layer of the patient, displaying, in the AR headset, a virtual portion of the object projected into the projected inner layer of the patient from the 3D data.

A method for augmenting real-time views of a patient with three-dimensional (3D) data, the method comprising:

• identifying 3D data for a patient, the 3D data including an outer layer of the patient and multiple inner layers of the patient;
• displaying, in an augmented reality (AR) headset, one of the inner layers of the patient from the 3D data projected onto real-time views of the outer layer of the patient, the real-time views being non-image actual views;
• generating, in the AR headset, a virtual user interface that includes options for altering the display of the projected inner layer of the patient from the 3D data;
• displaying, in the AR headset, the virtual user interface projected onto real-time views due to a focal orientation of the AR headset not being focused on the patient;
• and
• hiding, in the AR headset, the virtual user interface due to the focal orientation of the AR headset being focused on the patient.