Method and apparatus to provide updated patient images during robotic surgery

An apparatus for providing current images of a surgical site during a surgical procedure, the apparatus comprising:

• a control computer having at least one processor, the control computer configured to operate a medical imaging device to acquire three-dimensional (3D) data representing a volume of a patient on a patient table, the volume comprising anatomy to be manipulated during a surgical procedure by a surgical robot;
• and
• a display screen in communication with the at least one processor of the control computer,
• wherein the at least one processor of the control computer is configured to construct a 3D image of the volume from the 3D data,
• wherein the at least one processor is configured to construct a segmented 3D volumetric image of the anatomy from the 3D image of the volume,
• wherein the at least one processor is configured to bring a coordinate system of the surgical robot into registration with a coordinate system of the medical imaging device,
• wherein the at least one processor is configured to receive an electronic signal representing at least one force applied to the anatomy during the surgical procedure and detected by a force sensor of the surgical robot, wherein the at least one force produces a geometrical change of the anatomy,
• wherein the at least one processor is configured to automatically update the segmented 3D volumetric image of the anatomy, based on the at least one force, to produce an updated segmented 3D volumetric image of the anatomy that visually shows the geometrical change of the anatomy, and
• wherein the at least one processor is configured to display the updated segmented 3D volumetric image of the anatomy in real-time during the surgical procedure at the display screen.

A non-transitory, computer-readable data storage medium encoded with programming instructions, the storage medium being loaded into a computer of a surgery facility comprising a patient table configured to receive a patient thereon, a medical imaging device, and a surgical robot comprising an instrument that is operable during a surgical procedure, and a force sensor that detects a force applied by the instrument to anatomy of the patient that is involved in the surgical procedure, the programming instructions causing the computer to:

• receive three-dimensional (3D) data representing a volume of a patient on the patient table, the volume comprising an anatomy to be manipulated, during a surgical procedure, by a surgical robot;
• construct a 3D image of the volume from the 3D data;
• construct a segmented 3D volumetric image of the anatomy from the 3D image of the volume;
• bring a coordinate system of the surgical robot into registration with a coordinate system of the medical imaging device;
• generate an electronic signal representing at least one force applied to the anatomy during the surgical procedure and detected by a force sensor of the surgical robot, wherein the at least one force produces a geometrical change of the anatomy;
• automatically update the segmented 3D volumetric image of the anatomy, based on the at least one force, to produce an updated segmented 3D volumetric image of the anatomy that visually shows the geometrical change of the anatomy;
• and
• display, at a display screen, the updated segmented 3D volumetric image of the anatomy in real-time during the surgical procedure.

A medical device system comprising:

• a medical imaging system;
• a surgical robot;
• a display showing an updated segmented three-dimensional (3D) volumetric image of anatomy of a patient, the updated segmented 3D image of the anatomy showing in real time a geometrical change of the anatomy caused by the surgical robot during a surgical procedure;
• and
• a processor that executes coded instructions to:
• bring a coordinate system of the medical imaging device and a coordinate system of the surgical robot into registration;
• receive 3D data from the medical imaging device during the surgical procedure, the 3D data representing a volume of the patient comprising the anatomy;
• generate a 3D image of the volume from the 3D data;
• generate a first segmented 3D volumetric image of the anatomy from the 3D image of the volume;
• receive a signal representing a force on the anatomy by the surgical robot causing the geometrical change of the anatomy;
• and
• generate the updated segmented 3D volumetric image of the anatomy from the first segmented 3D volumetric image of the anatomy by using the registration and the signal representing the force on the anatomy.