Augmented reality patient positioning using an atlas

A non-transitory computer-readable storage medium for generating augmentation information relating to a medical image, using output of an augmented reality device having an output unit for outputting the augmentation information, comprising instructions, which when executed by at least one processor causes the at least one processor to:

• acquire patient medical image data which describes a patient image of an anatomical body part of the patient;
• acquire geometric model data which describes a geometric model of at least part of the patient's body;
• register the patient image with the geometric model;
• obtain, using the augmented reality device, a real image of a physical structure corresponding to at least part of:
• (1) the anatomical body part or the at least part of the patient's body, and (2) a medical device;
• output augmentation information using the output unit of the augmented reality device based on a result of registering the patient image with the geometric model, the augmentation information including at least a part of the geometric model or at least part of the patient image, overlaid on at least part of the real image acquired using the augmented reality device,
• wherein the output unit of the augmented reality device includes a display for displaying the augmentation information,
• wherein the augmented reality device includes a head-mounted distance measurement unit having a light source for emitting measurement light onto the physical structure to measure a distance between the augmented reality device and the physical structure, and
• wherein at least the part of the geometric model or at least part of the patient image describing at least part of the anatomical body part is registered, based on the measured distance between the augmented reality device and the physical structure, with the real image acquired using the augmented reality device;
• and
• determine, based on at least the part of the geometric model or at least part of the patient image describing at least part of the anatomical body part being registered with the real image of the physical structure, control data for controlling the medical device for changing the position of the physical structure to comply with a position of the physical structure defined by the augmentation information.

A medical system for generating augmentation information relating to a medical image, using output of an augmented reality device having an output unit for outputting the augmentation information, the medical system comprising:

• at least one computer having at least one processor with associated memory, the memory storing instructions which when executed, cause the at least one processor to:
• acquire patient medical image data which describes a patient image of an anatomical body part of the patient;
• acquire geometric model data which describes a geometric model of at least part of the patient's body;
• register the patient image with the geometric model;
• obtain, using the augmented reality device, a real image of a physical structure corresponding to at least part of:
• (1) the anatomical body part or the at least part of the patient's body, and (2) a medical device;
• output augmentation information using the output unit of the augmented reality device based on a result of registering the patient image with the geometric model, the augmentation information including at least a part of the geometric model or at least part of the patient image, overlaid on at least part of the real image acquired using the augmented reality device,
• wherein the output unit of the augmented reality device includes a display for displaying the augmentation information,
• wherein the augmented reality device includes a head-mounted distance measurement unit having a light source for emitting measurement light onto the physical structure to measure a distance between the augmented reality device and the physical structure, and
• wherein at least the part of the geometric model or at least part of the patient image describing at least part of the anatomical body part is registered, based on the measured distance between the augmented reality device and the physical structure, with the real image acquired using the augmented reality device;
• and
• determining, based on at least the part of the geometric model or at least part of the patient image describing at least part of the anatomical body part being registered with the real image of the physical structure, control data for controlling the medical device for changing the position of the physical structure to comply with a position of the physical structure defined by the augmentation information.