Integrated AI-powered adaptive robotic surgery system

A robotic surgical system for performing a surgical procedure at a surgical site of a patient, comprising:

• a surgeon console including a display configured to receive one or more multi-modal surgeon commands and present a real-time visual feedback to a surgeon during performance of a surgical procedure at a surgical site using one or more surgical instruments;
• a patient console coupled to the surgeon console;
• a surgical robot including one or more surgical arms to manipulate the one or more of the surgical instruments, each of the one or more surgical arms configured to have a control in one or more multiple degrees of freedom;
• an imaging system including one or sensors and one or cameras producing pre-operative images at the surgical target site for tissue to be modified or removed, tissue boundries, and critical structure, acquisition of the pre-operative images of the patient before a surgical procedure allowing a surgeon to visualize and create a modifiable surgical plan based on the patient's anatomy;
• an AI system coupled to or including a robotic surgery control system and a feedback loop, the AI system including one or more machine learning algorithms configured to:
• (i) process a baseline pre-operative images to generate an AI model and a virtual model including one or more boundary indicators delineating at least a boundary of the surgical target tissue and a boundary of a protected tissue comprising the one or more critical structures;
• (ii) receive via a touch-screen user interface of the surgeon console, a surgeon annotation that modifies at least one of the boundary indicators;
• (iii) register the virtual model to the intraoperative image data;
• and (iv) provide real-time guidance by causing the surgeon console display to present the intraoperative image data with the boundary indicators overlaid on the displayed intraoperative image data, and by generating at least one of an alert or an inhibition of instrument positioning when a current or predicted instrument position is outside the boundary of the surgical target tissue or within a threshold distance of the boundary of the protected tissue the AI model including detecting a surgeon information directed to at least one of:
• a surgeon skill level, a physiological state of the surgeon, a cognitive load of the surgeon;
• and
• a data integration module configured to interface with one or more of external hospital systems.

A robotic surgical system, comprising:

• a surgeon console operatively coupled to a patient console and one or more surgical instruments, the surgeon console configured for use by a surgeon to perform a surgical procedure at a surgical site using one or more surgical instruments;
• a surgeon computer coupled to or integrated with the surgeon console, the surgeon computer coupled further operatively connected to the one or more surgical instruments;
• an imaging system including a sensor array, the sensor array including one or more sensor types selected from the group including at least one of one or more:
• imaging devices, depth sensors, proximity sensors, 3D laser scanners, stereoscopic cameras, infrared cameras, ultrasonic sensors, electromagnetic tracking sensors, radar-based sensors, and physiological sensors, the sensor array configured to capture spatial and contextual data of a patient and an operating environment, the imaging system configured to acquire (i) baseline pre-operative images of at least a portion of the surgical site before initiation of tissue modification, including surgical target tissue to be modified or removed and one or more critical structures, and (ii) intraoperative image data during the surgical procedure, the baseline pre-operative images enabling the surgeon to create a modifiable surgical plan based on the patient's anatomy;
• a surgical robot with one or more surgical arms and operatively coupled to the surgeon console;
• an AI system coupled to or including a robotic surgery control system and a feedback loop, the AI system including one or more machine learning algorithms configured to:
• (i) process the baseline pre-operative images to generate an AI model and a virtual model including one or more boundary indicators delineating at least a boundary of the surgical target tissue and a boundary of a protected tissue comprising the one or more critical structures;
• (ii) receive, via a touch-screen user interface of the surgeon console, a surgeon annotation that modifies at least one of the boundary indicators;
• (iii) register the virtual model to the intraoperative image data;
• and (iv) provide real-time guidance by causing the surgeon console display to present the intraoperative image data with the boundary indicators overlaid on the displayed intraoperative image data, and by generating at least one of an alert or an inhibition of instrument positioning when a current or predicted instrument position is outside the boundary of the surgical target tissue or within a threshold distance of the boundary of the protected tissue.