Aligning pre-operative scan images to real-time operative images for a mediated-reality view of a surgical site

A non-transitory computer-readable storage medium storing instructions for generating a mediated reality view of a surgical site, the instructions when executed by a processor of a mediated reality system, cause the mediated reality system to:

• receive pre-operative images representing three-dimensional anatomy of a patient in a first position;
• receive post-positioning images captured after the patient is positioned for surgery in a second position;
• identify, based on the pre-operative images and the post-positioning images, a set of corresponding features visible in the pre-operative images and the post-positioning images;
• apply a first transformation to the pre-operative images to substantially align locations of the corresponding features in the pre-operative images to respective locations in the post-positioning images to generate initial transformed pre-operative images;
• capture, by a camera array, real-time images of the patient;
• overlay the initial transformed pre-operative images on the real-time images to generate an initial mediated reality view;
• and
• provide the initial mediated reality view to a display device for display.

A mediated reality system comprising:

• a camera array;
• a display device;
• a processor;
• and
• a non-transitory computer-readable storage medium storing instructions for generating a mediated reality view of a surgical site, the instructions when executed by a processor of the mediated reality system, cause the mediated reality system to:
• receive pre-operative images representing three-dimensional anatomy of a patient in a first position;
• receive post-positioning images captured after the patient is positioned for surgery in a second position;
• identify, based on the pre-operative images and the post-positioning images, a set of corresponding features visible in the pre-operative images and the post-positioning images;
• apply a first transformation to the pre-operative images to substantially align locations of the corresponding features in the pre-operative images to respective locations in the post-positioning images to generate initial transformed pre-operative images;
• capture, by the camera array, real-time images of the patient;
• overlay the initial transformed pre-operative images on the real-time images to generate an initial mediated reality view;
• and
• provide the initial mediated reality view to the display device for display.