Deep-learning-based real-time remaining surgery duration (rsd) estimation

A computer-implemented method for predicting in real-time a remaining surgical duration (RSD) of a live surgical session of a surgical procedure based on a real-time endoscope video of the live surgical session, the method comprising:

• sampling a set of N frames of the endoscope video corresponding to the elapsed portion of the live surgical session between a) the beginning of the endoscope video corresponding to the beginning of the live surgical session and b) a current frame corresponding to a current time of the live surgical session;
• feeding the set of N frames into a trained RSD machine-learning (ML) model for the surgical procedure;
• outputting a current RSD prediction from the trained RSD ML model based on the set of N frames, wherein the outputted current RSD prediction is part of a real-time RSD prediction sequence of the live surgical session;
• and
• predicting a delay based on determining a deviation, as the real-time RSD prediction sequence or a real-time RSD prediction curve deviates from a standard or ideal RSD prediction curve.

An article of manufacture comprising a machine-readable medium having stored instructions that configure a processor to predict in real-time a remaining surgical duration (RSD) of a live surgical session of a surgical procedure based on a real-time endoscope video of the live surgical session, by:

• sampling a set of N frames of the endoscope video corresponding to the elapsed portion of the live surgical session between a) the beginning of the endoscope video corresponding to the beginning of the live surgical session and b) a current frame corresponding to a current time of the live surgical session;
• feeding the set of N frames into a trained RSD machine-learning (ML) model for the surgical procedure;
• outputting a current RSD prediction from the trained RSD ML model based on the set of N frames, wherein the outputted current RSD prediction is part of a real-time RSD prediction sequence of the live surgical session;
• and
• predicting a delay based on determining a deviation, as the real-time RSD prediction sequence or a real-time RSD prediction curve deviates from a standard or ideal RSD prediction curve.

A surgical robotic system comprising:

• a processor;
• and
• memory that stores instructions which configure the processor to compute in real-time a remaining surgical duration (RSD) of a live surgical session of a surgical procedure based on a real-time endoscope video of the live surgical session, by:
• sampling a set of N frames of the endoscope video corresponding to the elapsed portion of the live surgical session between a) the beginning of the endoscope video corresponding to the beginning of the live surgical session and b) a current frame corresponding to a current time of the live surgical session;
• feeding the set of N frames into a trained RSD machine-learning (ML) model for the surgical procedure;
• outputting a current RSD prediction from the trained RSD ML model based on the set of N frames, wherein the outputted current RSD prediction is part of a real-time RSD prediction sequence of the live surgical session;
• and
• predicting a delay based on determining a deviation, as the real-time RSD prediction sequence or a real-time RSD prediction curve deviates from a standard or ideal RSD prediction curve.