Surgery evaluation using machine-learning-based surgical video analysis

A system for evaluating an execution of a surgical procedure, the system comprising:

• one or more processors;
• and
• a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the system to;
• receive a surgical video depicting the execution of the surgical procedure;
• segment the surgical video into a set of phase segments corresponding to a set of predefined phases of the surgical procedure;
• for each phase segment corresponding to a given phase in the set of predefined phases,
• detect a surgical tool associated with the surgical procedure in the phase segment as output of a first class of machine-learning classifiers in response to the phase segment as input;
• compute a tool-related score based on the detected surgical tool;
• detect a surgical event associated with the surgical procedure in the phase segment as output of a second class of machine-learning classifiers in response to the phase segment as input;
• compute an event-related score based on the detected surgical event;
• and
• compute a performance score for the phase segment by combining the tool-related score with the event-related score;
• generate an overall performance score for the execution of the surgical procedure based on the set of computed performance scores corresponding to the set of predefined phases;
• produce an annotated version of the surgical video by annotating the surgical video based on the overall performance score;
• and
• using the annotated version of the surgical video to train at least one of the first class of machine-learning classifiers and the second class of machine-learning classifiers to detect one or more surgical tools and one or more surgical events, respectively.

An apparatus for evaluating an execution of a surgical procedure, comprising:

• a data-processing module comprising a non-transitory computer-readable medium comprising instructions that, when executed, cause the apparatus to:
• receive a surgical video depicting the execution of the surgical procedure;
• segment the surgical video into a set of phase segments corresponding to a set of predefined phases of the surgical procedure;
• for each phase segment corresponding to a given phase in the set of predefined phases,
• detect a surgical tool associated with the surgical procedure in the phase segment as output of a first class of machine-learning classifiers in response to the phase segment as input;
• compute a tool-related score based on the detected surgical tool;
• detect a surgical event associated with the surgical procedure in the phase segment as output of a second class of machine-learning classifiers in response to the phase segment as input;
• compute an event-related score based on the detected surgical event;
• and
• compute a performance score for the phase segment by combining the tool-related score with the event-related score;
• generate an overall performance score for the execution of the surgical procedure based on the set of computed performance scores corresponding to the set of predefined phases;
• produce an annotated version of the surgical video by annotating the surgical video based on the overall performance score;
• and
• using the annotated version of the surgical video to train at least one of the first class of machine-learning classifiers and the second class of machine-learning classifiers to detect one or more surgical tools and one or more surgical events, respectively.