Augmented reality guidance methods and systems for use with microscopes or endoscopes in surgical procedures

A computer-implemented method comprising:

• tracking a microscope or an endoscope in a coordinate system,
• receiving at least one image of a physical target tissue from an imaging system different than the microscope or endoscope,
• wherein the physical target tissue is not visible through the microscope or endoscope,
• registering the at least one image in the coordinate system,
• generating a virtual display of the physical target tissue,
• wherein the virtual display comprises three-dimensional digital representation of the at least one image of the physical target tissue,
• tracking a patient, a physical tissue visible through the microscope or endoscope, the physical target tissue or a combination thereof in the coordinate system,
• wherein the physical target tissue and the physical tissue are different,
• generating an augmented view, the augmented view comprising the virtual display, displaying the augmented view at the coordinates of the physical target tissue in the coordinate system;
• and
• displaying the physical target tissue outside a field of view of the microscope or endoscope,
• wherein a portion of the augmented view extends outside the field of view of view of the microscope or endoscope.

A surgical method comprising:

• placing an access, a portal, an access portal, a tube, a retractor, or a combination thereof in a patient,
• positioning and/or advancing a microscope, an endoscope, a surgical tool, a surgical instrument, a device, an implant, or a combination thereof in relationship to the access, a portal, an access portal, a tube, a retractor, or a combination thereof in a patient,
• tracking the microscope or endoscope in a coordinate system,
• receiving at least one image of a physical target tissue from an imaging system different than the microscope or endoscope,
• wherein the physical target tissue is not visible through the microscope or endoscope,
• registering the at least one image in the coordinate system,
• generating a virtual display of the physical target tissue,
• wherein the virtual display comprises a three-dimensional digital representation of the at least one image of the physical target tissue,
• tracking a patient, a physical tissue visible through the microscope or endoscope, the physical target tissue or a combination thereof in the coordinate system, wherein the physical target tissue and the physical tissue are different,
• generating an augmented view, the augmented view comprising the virtual display,
• displaying the augmented view at the coordinates of the physical target tissue in the coordinate system,
• displaying the physical target tissue outside a field of view of the microscope or endoscope,
• wherein a portion of the augmented view extends outside the field of view of view of the microscope or endoscope,
• generating a virtual display comprising a 2D or a 3D representation of a virtual tool, virtual instrument, virtual implant, virtual device or a combination thereof, wherein the virtual tool, virtual instrument, virtual implant, virtual device or a combination thereof correspond to a physical tool, physical instrument, physical implant, physical device or a combination thereof, and
• tracking the surgical tool, surgical instrument, device, implant or combination thereof in the coordinate system.