Registration of 3D and 2D images for surgical navigation and robotic guidance without using radiopaque fiducials in the images

A system for registering a 3D image volume comprising:

• a portable imaging system having an imaging tracking marker array (ITMA);
• a dynamic reference base (DRB) including patient tracking markers and attachable to a patient;
• a tracking device configured to track a position and orientation of both the ITMA and the DRB simultaneously;
• a surgical robot configured to assist in a surgical operation of the patient and be in communication with the imaging system, the surgical robot including a base, a robot arm coupled to the base and an end effector coupled to the robot arm;
• a processor adapted to:
• receive first and second 2D images at different orientations that have been captured intra-operatively by the imaging system;
• receive corresponding optical images of the patient from the tracking device at a time of image capture, the optical images containing the ITMA and the DRB;
• register the first and second 2D images to the patient using the captured images from the imaging system and the optical images containing the ITMA and the DRB;
• generate a customized 3D image volume from a generic 3D image volume based on the first and second 2D images;
• merge the 2D images to the customized 3D image volume so as to register the customized 3D image volume to the patient based on a registration information of the 2D images;
• and
• move the end effector to a position for receiving a surgical instrument along a planned trajectory based on the registration information of the customized 3D image volume,
• wherein the processor is configured to register the first 2D image to the patient by:
• determining transform B representing a pose of the first 2D image in an imaging space relative to the imaging system in a physical space based on the received optical images.