# US 2019/0246088 A1 · Augmented Reality Guidance for Spinal Surgery and Spinal Procedures

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0246088 A1
- Document ID: US20190246088A1
- Status: Published application
- Kind: A1
- Application number: 16/240,937
- Filed: 2019-01-07
- Published: 2019-08-08
- Earliest claimed: 2014-12-30
- Assignees: OnPoint Medical, Inc.
- Applicants: OnPoint Medical, Inc.
- Inventors: Carlos Quiles Casas
- Web page: https://patentgrep.com/patent/US20190246088A1

## CPC

- H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- G02B 27/0172: {characterised by optical features}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04N 13/156: Mixing image signals
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- H04N 13/279: the virtual viewpoint locations being selected by the viewers or determined by tracking
- H04N 13/296: Synchronisation thereof; Control thereof
- H04N 13/366: using viewer tracking
- A61B 1/00: Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes (examination of body cavities or body tracts using ultrasonic, sonic or infrasonic waves A61B8/12; endoscopic instruments for taking cell samples or for biopsy A61B10/04); Illuminating arrangements therefor (for the eyes A61B3/00)
- A61B 2034/2057: {Details of tracking cameras}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- A61B 2090/371: {with simultaneous use of two cameras}
- G02B 2027/0134: {of stereoscopic type}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G06F 2203/04804: Transparency, e.g. transparent or translucent windows
- G06T 2210/41: Medical
- G16Z 99/00: Subject matter not provided for in other main groups of this subclass
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16Z: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS, NOT OTHERWISE PROVIDED FOR

## Abstract

Embodiments disclose a real-time surgery method and apparatus for displaying a stereoscopic augmented view of a patient from a static or dynamic viewpoint of the surgeon, which employs real-time three-dimensional surface reconstruction for preoperative and intraoperative image registration. Stereoscopic cameras provide real-time images of the scene including the patient. A stereoscopic video display is used by the surgeon, who sees a graphical representation of the preoperative or intraoperative images blended with the video images in a stereoscopic manner through a see through display.

## Claims

### Claim 1

A system comprising:
a stereoscopic optical see-through head mounted display;
at least one computing system configured to receive preoperative, intraoperative or preoperative and intraoperative image information of the physical spine of a patient,
at least one marker configured to be attached to the patient,
wherein the at least one computing system is configured to register the image information of the physical spine in a common coordinate system,
wherein the at least one computing system is configured to register the at least one marker in the common coordinate system,
wherein the at least one computing system is configured to generate a stereoscopic display by the optical see-through head mounted display based on the image information of the physical spine,
wherein the at least one computing system is configured to track movement from a first position to a second position of the physical spine during surgery,
wherein the at least one computing system is configured to track the location and orientation of the optical see-through head mounted display in the common coordinate system,
wherein the at least one computing system is configured to superimpose the stereoscopic display on a target portion of the physical spine visible directly through the stereoscopic optical see-through head mounted display,
wherein the at least one computing system is configured to adjust in real time the stereoscopic display based on the location and orientation of the optical see-through head mounted display,
wherein the at least one computing system is configured to adjust in real time the stereoscopic display based on the tracked movement from the first position to the second position of the physical spine,
and
wherein the first and second positions of the physical spine of the patient are different.

### Claim 2 (depends on 1)

The system of claim 1, wherein the at least one computing system is configured to reconstruct one or more preoperative, intraoperative or preoperative and intraoperative two-dimensional (2D) images into three-dimensional (3D) images using 2D to 3D reconstruction.

### Claim 3 (depends on 1)

The system of claim 1, wherein the at least one computing system is configured to track movement from the first position to the second position of the physical spine by tracking the position of the at least one markers in real time.

### Claim 4 (depends on 1)

The system of claim 1, wherein the at least one computing system is configured to compare tracking data of one or more instruments and/or one or more devices to the tracked position of the physical spine during surgery in real time.

### Claim 5 (depends on 1)

The system of claim 1, wherein the at least one computing system is configured to track the location and/or orientation of one or more instruments, one or more devices, the physical spine, the surgeon, the optical see-through head mounted display or combinations thereof in real time.

### Claim 6

(canceled)

### Claim 7

(canceled)

### Claim 8

(canceled)

### Claim 9

(canceled)

### Claim 10

(canceled)

### Claim 11

(canceled)

### Claim 12 (depends on 1)

The system of claim 1, wherein the image information of the physical spine of the patient is registered with one or more structures of the physical spine using a navigation system.

### Claim 13 (depends on 1)

The system of claim 1, wherein the at least one computing system is configured to provide for adjustment of transparency, color and contrast of the stereoscopic display displayed by the optical see-through head mounted display superimposed on the physical spine.

### Claim 14 (depends on 1)

The system of claim 1, wherein the at least one computing system is configured to display one or more virtual graphical representations of tracked instruments or devices by the optical see-through head mounted display superimposed on the physical spine.

### Claim 15 (depends on 1)

The system of claim 1, wherein the stereoscopic display of the spine displayed by the optical see-through head mounted display includes images of individually rendered vertebrae registered in the common coordinate system superimposed with individual vertebra of the physical spine.

### Claim 16 (depends on 15)

The system of claim 15, wherein the at least one computing system is configured to track pose changes of the individual vertebrae in the common coordinate system, and wherein the pose changes are translated into display of a different second stereoscopic display at a location and orientation that corresponds to the location and orientation of the individual vertebrae of the patient's spine after the pose changes.

### Claim 17 (depends on 5)

The system of claim 5, wherein the tracking of the one or more instruments, the one or more devices, the physical spine, the surgeon, the optical see-through head mounted display or combinations thereof is performed using at least one attached passive optical markers, active optical markers, and/or at least one of magnetic tracking, electromagnetic tracking, ultrasonic tracking, mechanical tracking, tracking using inertial measurement units and 3D scanners.

### Claim 18 (depends on 1)

The system of claim 1, wherein the optical see-through head mounted display is configured to display at least one of a graphical representations of one or more instruments, surgical guides or techniques, or anatomical models superimposed on the physical spine.

### Claim 19

A system comprising:
a stereoscopic optical see-through head mounted display;
at least one computing system configured to receive preoperative, intraoperative, or preoperative and intraoperative image information from two or more images of individual vertebrae of a patient; and
at least one marker configured to be attached to the patient,
wherein the at least one computing system is configured to register the two or more images of the individual vertebrae in a common coordinate system,
wherein the at least one computing system is configured to register the at least one markers in the common coordinate system,
wherein the at least one computing system-is configured to generate a stereoscopic display by the optical see-through head mounted display based on the two or more images of the individual vertebrae,
wherein the at least one computing system is configured to track movement from a first position to a second position of the physical spine during surgery by tracking the position of the at least one markers,
wherein the at least one computing system is configured to track the location and orientation of the optical see-through head mounted display in the common coordinate system,
wherein the at least one computing system is configured to superimpose the stereoscopic display of the two or more images of the individual vertebrae on corresponding individual vertebrae of the physical spine visible directly through the stereoscopic optical see-through head mounted display,
wherein the at least one computing system is configured to adjust in real time the stereoscopic display of the two or more images of the individual vertebrae based on the location and orientation of the optical see-through head mounted display,
wherein the at least one computing system is configured to adjust in real time the stereoscopic display of the two or more images of the individual vertebrae based on the tracked movement from the first position to the second position of the physical spine, and
wherein the first and second positions of the physical spine of the patient are different.

### Claim 20

A system comprising:
a stereoscopic optical see-through head mounted display;
at least one computing system configured to receive preoperative, intraoperative, or preoperative and intraoperative image information from two or more images of two or more individual vertebrae of a patient for stereoscopic display by the optical see-through head mounted display; and
at least two markers configured to be attached to the patient,
wherein the at least one computing system is configured to register the two or more images of the two or more individual vertebrae in a common coordinate system,
wherein the at least one computing system is configured to register the at least two markers in the common coordinate system,
wherein the at least one computing system is configured to generate a stereoscopic display by the optical see-through head mounted display based on the two or more images of the two or more individual vertebrae,
wherein the at least one computing system is configured to track movement from a first position to a second position of two or more individual vertebrae of the physical spine during surgery by tracking the position of the at least two markers,
wherein the at least one computing system is configured to track the location and orientation of the optical see-through head mounted display in the common coordinate system,
wherein the at least one computing system is configured to superimpose the stereoscopic display of the two or more images of the two or more individual vertebrae on the corresponding individual vertebrae of the physical spine visible directly through the stereoscopic optical see-through head mounted display,
wherein the at least one computing system is configured to adjust in real time the stereoscopic display based on the location and orientation of the optical see-through head mounted display,
wherein the at least one computing system is configured to adjust in real time the stereoscopic display of the two or more images of the two or more individual vertebrae based on the tracked movement from the first position to the second position of the corresponding individual vertebrae of the physical spine,
and
wherein the first and second positions of the physical spine of the patient are different.

### Claim 21 (depends on 1)

The system of claim 1, wherein the image information comprises a 2D image of the spine of the patient.

### Claim 22 (depends on 21)

The system of claim 21, wherein the 2D image of the spine of the patient comprises a CT image, an MRI image, an ultrasound image, an x-ray image or combinations thereof.

### Claim 23 (depends on 1)

The system of claim 1, wherein the image information comprises is a 3D image of the spine of the patient.

### Claim 24 (depends on 23)

The system of claim 23, wherein the 3D image of the spine is generated using a CT image, an MRI image, an ultrasound image, an x-ray image or combinations thereof.

### Claim 25 (depends on 1)

The system of claim 1, wherein the tracked movement from the first position to the second position of the physical spine during surgery is a rotation of one or more vertebrae.

### Claim 26 (depends on 1)

The system of claim 1, wherein the tracked movement from the first position to the second position of the physical spine during surgery is a lateralization of one or more vertebrae.

### Claim 27 (depends on 1)

The system of claim 1, wherein the tracked movement from the first position to the second position of the physical spine during surgery is a change in sagittal position of one or more vertebrae.

### Claim 28 (depends on 1)

The system of claim 1, wherein the at least one marker is configured to be attached to an exposed portion of the physical spine.

### Claim 29 (depends on 1)

The system of claim 1, wherein the intraoperative image information is obtained using a CT scan.

### Claim 30 (depends on 1)

The system of claim 1, wherein the at least one computing system comprises the computing system configured to receive preoperative, intraoperative or preoperative and intraoperative image information from the patient, the computing system configured to register the image information of the physical spine in a common coordinate system, the computing system configured to register the at least one-marker in the common coordinate system, the computing system configured to generate a stereoscopic display by the optical see-through head mounted display based on the image of the spine, the computing system configured to track movement from a first position to a second position of the physical spine during surgery, the computing system configured to track the location and orientation of the optical see-through head mounted display in the common coordinate system, the computing system configured to adjust in real time the stereoscopic display based on the location and orientation of the optical see-through head mounted display, the computing system configured to adjust in real time the stereoscopic display based on the tracked movement from the first position to the second position of the physical spine, the computing system configured to superimpose the stereoscopic display on the target portion of the physical spine visible directly through the stereoscopic optical see-through head mounted display, and wherein one or more of the at least one computing system are the same.

### Claim 31 (depends on 1)

The system of claim 1, wherein the at least one computing system comprises the computing system configured to receive preoperative, intraoperative or preoperative and intraoperative image information from the patient, the computing system configured to register the image information of the physical spine in a common coordinate system, the computing system configured to register the at least one-marker in the common coordinate system, the computing system configured to generate a stereoscopic display by the optical see-through head mounted display based on the image of the spine, the computing system configured to track movement from a first position to a second position of the physical spine during surgery, the computing system configured to track the location and orientation of the optical see-through head mounted display in the common coordinate system, the computing system configured to adjust in real time the stereoscopic display based on the location and orientation of the optical see-through head mounted display, the computing system configured to adjust in real time the stereoscopic display based on the tracked movement from the first position to the second position of the physical spine, the computing system configured to superimpose the stereoscopic display on the target portion of the physical spine visible directly through the stereoscopic optical see-through head mounted display, and wherein one or more of the at least one computing system are different.

### Claim 32 (depends on 19)

The system of claim 19, wherein the at least one computing system is configured to reconstruct one or more preoperative, intraoperative or preoperative and intraoperative two-dimensional images into three-dimensional images using 2D to 3D reconstruction.

### Claim 33 (depends on 19)

The system of claim 19, wherein the at least one computing system is configured to track movement from the first position to the second position of the physical spine by tracking the position of the at least one marker in real time.

### Claim 34 (depends on 19)

The system of claim 19, wherein the at least one computing system is configured to compare tracking data of one or more instruments and/or one or more devices to the tracked position of the physical spine during surgery in real time.

### Claim 35 (depends on 19)

The system of claim 19, wherein the at least one computing system is configured to track the location and/or orientation of one or more instruments, one or more devices, the physical spine, the surgeon, the optical see-through head mounted display or combinations thereof in real time.

### Claim 36 (depends on 19)

The system of claim 19, wherein the image information of the two or more images of individual vertebrae is registered with one or more structures of the physical spine using a navigation system.

### Claim 37 (depends on 19)

The system of claim 19, wherein the at least one computing system is configured to provide for adjustment of transparency, color and contrast of the stereoscopic display displayed by the optical see-through head mounted display superimposed on the physical spine.

### Claim 38 (depends on 19)

The system of claim 19, wherein the at least one computing system is configured to display one or more virtual graphical representations of tracked instruments or devices by the optical see-through head mounted display superimposed on the individual vertebrae of the physical spine.

### Claim 39 (depends on 19)

The system of claim 19, wherein the at least one computing system is configured to track pose changes of the individual vertebrae in the common coordinate system and wherein the pose changes are translated into display of a different second stereoscopic display at a location and orientation that corresponds to the location and orientation of the individual vertebrae of the spine of the patient after the pose changes.

### Claim 40 (depends on 35)

The system of claim 35, wherein the tracking of the one or more instruments, one or more devices, the physical spine, the surgeon, the optical see-through head mounted display or combinations thereof is performed using at least one attached passive optical marker, active optical marker, and/or at least one of magnetic tracking, electromagnetic tracking, ultrasonic tracking, mechanical tracking, tracking using inertial measurement units and 3D scanners.

### Claim 41 (depends on 19)

The system of claim 19, wherein the optical see-through head mounted display is configured to display at least one of a graphical representation of one or more instruments, surgical guides or techniques, or anatomical models superimposed on the physical spine.

### Claim 42 (depends on 19)

The system of claim 19, wherein the two or more images comprise at least one 2D image of the individual vertebrae of the patient.

### Claim 43 (depends on 42)

The system of claim 42, wherein the 2D image of the spine of the patient comprises a CT image, an MRI image, an ultrasound image, an x-ray image, or combinations thereof.

### Claim 44 (depends on 19)

The system of claim 19, wherein the two or more images comprise at least one 3D image of the spine of the patient.

### Claim 45 (depends on 44)

The system of claim 44, wherein the 3D image of the spine is generated using a CT image, an MRI image, an ultrasound image, an x-ray image, or combinations thereof.

### Claim 46 (depends on 19)

The system of claim 19, wherein the tracked movement from the first position to the second position of the physical spine during surgery is a rotation of one or more vertebrae.

### Claim 47 (depends on 19)

The system of claim 19, wherein the tracked movement from the first position to the second position of the physical spine during surgery is a lateralization of one or more vertebrae.

### Claim 48 (depends on 19)

The system of claim 19, wherein the tracked movement from the first position to the second position of the physical spine during surgery is a change in sagittal position of one or more vertebrae.

### Claim 49 (depends on 19)

The system of claim 19, wherein the at least one marker is configured to be attached to an exposed portion of the physical spine.

### Claim 50 (depends on 19)

The system of claim 19, wherein the intraoperative image information is obtained from the patient using a CT scan.

### Claim 51 (depends on 19)

The system of claim 19, wherein the at least one computing system comprises the computing system configured to receive preoperative, intraoperative, or preoperative and intraoperative image information from two or more images of individual vertebrae of the patient, the computing system configured to register the two or more images of the individual vertebrae in the common coordinate system, the computing system configured to register the at least one marker in the common coordinate system, the computing system configured to generate the stereoscopic display by the optical see-through head mounted display based on the two or more images of the individual vertebrae, the computing system configured to track movement from a first position to a second position of the physical spine during surgery by tracking the position of the at least one marker, the computing system configured to track the location and orientation of the optical see-through head mounted display in the common coordinate system, the computing system configured to adjust in real time the stereoscopic display of the two or more images of the individual vertebrae based on the location and orientation of the optical see-through head mounted display, the computing system configured to adjust in real time the stereoscopic display of the two or more images of the individual vertebrae based on the tracked movement from the first position to the second position of the physical spine, the computing system configured to superimpose the stereoscopic display of the two or more images of the individual vertebrae on corresponding individual vertebrae of the physical spine visible directly through the stereoscopic optical see-through head mounted display, and wherein one or more of the at least one computing system are the same.

### Claim 52 (depends on 19)

The system claim 19, wherein the at least one computing system comprises the computing system configured to receive preoperative, intraoperative, or preoperative and intraoperative image information from two or more images of individual vertebrae of the patient, the computing system configured to register the two or more images of the individual vertebrae in the common coordinate system, the computing system configured to register the at least one marker in the common coordinate system, the computing system configured to generate the stereoscopic display by the optical see-through head mounted display based on the two or more images of the individual vertebrae, the computing system configured to track movement from a first position to a second position of the physical spine during surgery by tracking the position of the at least one marker, the computing system configured to track the location and orientation of the optical see-through head mounted display in the common coordinate system, the computing system configured to adjust in real time the stereoscopic display of the two or more images of the individual vertebrae based on the location and orientation of the optical see-through head mounted display, the computing system configured to adjust in real time the stereoscopic display of the two or more images of the individual vertebrae based on the tracked movement from the first position to the second position of the physical spine, the computing system configured to superimpose the stereoscopic display of the two or more images of the individual vertebrae on corresponding individual vertebrae of the physical spine visible directly through the stereoscopic optical see-through head mounted display, and wherein one or more of the at least one computing system are different.

### Claim 53 (depends on 20)

The system of claim 20, wherein the at least one computing system is configured to reconstruct one or more preoperative, intraoperative or preoperative and intraoperative two-dimensional images into three-dimensional images using 2D to 3D reconstruction.

### Claim 54 (depends on 20)

The system of claim 20, wherein the at least one computing system is configured to track movement from the first position to the second position of the two or more individual vertebrae of the physical spine during surgery by tracking the position of the at least two markers in real time.

### Claim 55 (depends on 20)

The system of claim 20, wherein the at least one computing system is configured to compare tracking data of one or more instruments and/or one or more devices to the tracked position of the two or more individual vertebrae of the physical spine during surgery in real time.

### Claim 56 (depends on 20)

The system of claim 20, wherein the at least one computing system is configured to track the location and/or orientation of one or more instruments, one or more devices, the physical spine, the surgeon, the optical see-through head mounted display in real time or combinations thereof.

### Claim 57 (depends on 20)

The system of claim 20, wherein the image information of two or more individual vertebrae of the patient is registered with one or more structures of the two or more individual vertebrae of the physical spine using a navigation system.

### Claim 58 (depends on 20)

The system of claim 20, wherein the at least one computing system is configured to provide for adjustment of transparency, color and contrast of the stereoscopic display displayed by the optical see-through head mounted display superimposed on the physical spine.

### Claim 59 (depends on 20)

The system of claim 20, wherein the at least one computing system is configured to display one or more virtual graphical representations of tracked instruments or devices by the optical see-through head mounted display superimposed on the two or more individual vertebrae.

### Claim 60 (depends on 20)

The system of claim 20, wherein the at least one computing system is configured to track pose changes of the individual vertebrae in the common coordinate system, and wherein the pose changes are translated into display of a different second stereoscopic display at a location and orientation that corresponds to the location and orientation of the individual vertebrae of the spine of the patient after the pose changes.

### Claim 61 (depends on 56)

The system of claim 56, wherein the tracking of the one or more instruments, one or more devices, the patient's spine, the surgeon, or the optical see-through head mounted display or combinations thereof is performed using at least one attached passive optical marker, active optical marker, and/or at least one of magnetic tracking, electromagnetic tracking, ultrasonic tracking, mechanical tracking, tracking using inertial measurement units and 3D scanners.

### Claim 62 (depends on 20)

The system of claim 20, wherein the optical see-through head mounted display is configured to display at least one of a graphical representation of one or more instruments, surgical guides or techniques, or anatomical models superimposed on the physical spine.

### Claim 63 (depends on 20)

The system of claim 20, wherein the two or more images comprise at least one 2D image of the individual vertebrae of the patient.

### Claim 64 (depends on 63)

The system of claim 63, wherein the 2D image of the individual vertebrae of the patient comprises a CT image, an MM image, an ultrasound image, an x-ray image or combinations thereof.

### Claim 65 (depends on 20)

The system of claim 20, wherein the two or more images of the individual vertebrae comprise at least one 3D image of the spine of the patient.

### Claim 66 (depends on 65)

The system of claim 65, wherein the 3D image of the individual vertebrae is generated using a CT image, an MRI image, an ultrasound image, an x-ray image or combinations thereof.

### Claim 67 (depends on 20)

The system of claim 20, wherein the tracked movement from the first position to the second position of the two or more individual vertebrae of the physical spine during surgery is a rotation of one or more vertebrae.

### Claim 68 (depends on 20)

The system of claim 20, wherein the tracked movement from the first position to the second position of the two or more individual vertebrae of the physical spine during surgery is a lateralization of one or more vertebrae.

### Claim 69 (depends on 20)

The system of claim 20, wherein the tracked movement from the first position to the second position of the two or more individual vertebrae of the physical spine during surgery is a change in sagittal position of one or more vertebrae.

### Claim 70 (depends on 20)

The system of claim 20, wherein the at least two markers are configured to be attached to an exposed portion of the physical spine.

### Claim 71 (depends on 20)

The system of claim 20, wherein the intraoperative image information is obtained from the patient using a CT scan.

### Claim 72 (depends on 20)

The system of claim 20, wherein the at least one computing system comprises the computing system configured to receive preoperative, intraoperative, or preoperative and intraoperative image information from two or more images of two or more individual vertebrae of a patient for stereoscopic display by the optical see-through head mounted display, the computing system configured to register the two or more images of the two or more individual vertebrae in a common coordinate system, the computing system configured to register the at least two markers in the common coordinate system, the computing system configured to generate the stereoscopic display by the optical see-through head mounted display based on the two or more images of the two or more individual vertebrae, the computing system configured to track movement from a first position to a second position of two or more individual vertebrae of the physical spine during surgery by tracking the position of the at least two markers, the computing system configured to track the location and orientation of the optical see-through head mounted display in the common coordinate system, the computing system configured to adjust in real time the stereoscopic display based on the location and orientation of the optical see-through head mounted display, the computing system configured to adjust in real time the stereoscopic display of the two or more images of the two or more individual vertebrae based on the tracked movement from the first position to the second position of the corresponding individual vertebrae of the physical spine, the computing system configured to superimpose the stereoscopic display of the two or more images of the two or more individual vertebrae on the corresponding individual vertebrae of the physical spine visible directly through the stereoscopic optical see-through head mounted display, and wherein one or more of the at least one computing system are the same.

### Claim 73 (depends on 20)

The system of claim 20, wherein the at least one computing system comprises the computing system configured to receive preoperative, intraoperative, or preoperative and intraoperative image information from two or more images of two or more individual vertebrae of a patient for stereoscopic display by the optical see-through head mounted display, the computing system configured to register the two or more images of the two or more individual vertebrae in a common coordinate system, the computing system configured to register the at least two markers in the common coordinate system, the computing system configured to generate the stereoscopic display by the optical see-through head mounted display based on the two or more images of the two or more individual vertebrae, the computing system configured to track movement from a first position to a second position of two or more individual vertebrae of the physical spine during surgery by tracking the position of the at least two markers, the computing system configured to track the location and orientation of the optical see-through head mounted display in the common coordinate system, the computing system configured to adjust in real time the stereoscopic display based on the location and orientation of the optical see-through head mounted display, the computing system configured to adjust in real time the stereoscopic display of the two or more images of the two or more individual vertebrae based on the tracked movement from the first position to the second position of the corresponding individual vertebrae of the physical spine, the computing system configured to superimpose the stereoscopic display of the two or more images of the two or more individual vertebrae on the corresponding individual vertebrae of the physical spine visible directly through the stereoscopic optical see-through head mounted display, and wherein one or more of the at least one computing system are different.
