Surgical image providing method using surgical robot, and information processing system

A system for a surgical robot comprising a plurality of articulated arms and a hand controller with a pair of movable members for opening and closing a tip of a surgical instrument attached to the articulated arm, the system comprising:

• a processor;
• and
• a memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the processor to:
• obtain images captured by an endoscope during a surgical procedure on a patient using the surgical robot, wherein the image shows a surgical site and the tip of the surgical instrument;
• obtain signals generated by a sensor during the surgical procedure, wherein the signal is generated according to a manipulation of the tip;
• and
• display a graphical user interface on a display, the graphical user interface being configured to display:
• a video generated from the obtained images, wherein the video shows the manipulation of the tip during the surgical procedure;
• a plurality of timeline regions respectively corresponding to the plurality of articulated arms, each of the timeline regions linearly extending in correspondence with a timeline of the surgical procedure, wherein the timeline region visually distinguishes a first duration that a first type of the surgical instrument is attached to the articulated arm, and a second duration that a second type of the surgical instrument is attached to the articulated arm;
• and
• a quantitative value generated based on at least one of the signals, wherein the quantitative value corresponds to a force applied to a tissue of the patient by the tip,
• wherein the processor displays, on the graphical user interface, the quantitative value in association with the manipulation shown in the video.

A computer-implemented method for a surgical robot comprising a plurality of articulated arms and a hand controller with a pair of movable members for opening and closing a tip of a surgical instrument attached to the articulated arm, the method comprising:

• obtaining images captured by an endoscope during a surgical procedure on a patient using the surgical robot, wherein the image shows a surgical site and the tip of the surgical instrument;
• obtaining signals generated by a sensor during the surgical procedure, wherein the signal is generated according to a manipulation of the tip;
• and
• displaying a graphical user interface on a display, the graphical user interface being configured to display:
• a video generated from the obtained images, wherein the video shows the manipulation of the tip during the surgical procedure;
• a plurality of timeline regions respectively corresponding to the plurality of articulated arms, each of the timeline regions linearly extending in correspondence with a timeline of the surgical procedure, wherein the timeline region visually distinguishes a first duration that a first type of the surgical instrument is attached to the articulated arm, and a second duration that a second type of the surgical instrument is attached to the articulated arm;
• and
• a quantitative value generated based on at least one of the signals, wherein the quantitative value corresponds to a force applied to a tissue of the patient by the tip,
• wherein the graphical user interface displays the quantitative value in association with the manipulation shown in the video.