System and method for fused image based navigation with late marker placement

A system for image guidance comprising:

• an image processing system;
• a first imaging module, including one or more cameras, in communication with the image processing system, wherein the image processing system is configured to:
• receive secondary image data from a second imaging device;
• segment said secondary image data to determine a first surface of interest;
• receive real-time imaging data from the first imaging module, wherein the real-time imaging data from the one or more cameras includes images of one or more of:
• a second surface of interest, an anchor marker, wherein the anchor marker provides a point and an orientation of the surface, a plurality of early or late markers, and tools;
• calculate a registration transform of the first surface of interest relative to the second surface of interest using one or more of:
• the anchor marker, the plurality of early or late markers, or the second surface;
• calculate a tracking location of the first imaging module relative to the first surface of interest using one or more of:
• the anchor marker, the plurality of early or late markers, the second surface, or features on the second surface;
• and
• create an enhanced image by combining information from the real-time imaging data with corresponding information computed from secondary image data based on the location of the first imaging module.