System and method for surgical performance tracking and measurement

A system for a machine learning architecture for surgical performance tracking and measurement, the system comprising:

• at least one memory storing the machine learning architecture for estimating a value related to surgical instrument motion, the machine learning architecture defined by a plurality of parameters representative of a surgical instrument instance segmentation architecture, a decomposition model, and a sequential relation architecture;
• at least one processor in communication with the at least one memory, the at least one processor configured to:
• receive a first data set comprising one or more surgical procedure video data sets comprising a plurality of frames, the plurality of frames comprising at least one surgical instrument, and label data for each of the one or more surgical procedure video data sets;
• train the machine learning architecture based on the first data set by:
• for each surgical procedure video data set in the first data set:
• processing each frame in the respective plurality of frames in the respective surgical procedure video data set with the plurality of parameters representative of the surgical instrument instance segmentation architecture to assign each pixel in the respective frame a likelihood of belonging to at least one surgical instrument to generate a respective mask for each surgical instrument in the respective frame;
• processing each of the generated masks within the respective frames with the plurality of parameters representative of the decomposition model to generate key point mask characteristics based on extracting surgical instrument characteristics from each mask in the respective frame;
• processing each of the key point mask characteristics in each frame with the plurality of parameters representative of the sequential relation architecture to generate the value related to surgical instrument motion;
• updating the plurality of parameters based on a loss value, the loss value based on a comparison of the respective label and the value related to surgical instrument motion;
• and
• store the trained machine learning architecture in the at least one memory.

A non-transitory computer readable medium storing machine interpretable instructions, which when executed, cause one or more processors to perform one or more steps corresponding to a method comprising:

• receiving a first data set comprising one or more surgical procedure video data sets comprising a plurality of frames, the plurality of frames comprising at least one surgical instrument, and label data for each of the one or more surgical procedure video data sets;
• training a machine learning architecture for estimating a value related to surgical instrument motion, the machine learning architecture defined by a plurality of parameters representative of a surgical instrument instance segmentation architecture, a decomposition model, and a sequential relation architecture by;
• for each surgical procedure video data set in the first data set:
• processing each frame in the respective plurality of frames in the respective surgical procedure video data set with the plurality of parameters representative of the surgical instrument instance segmentation architecture to assign each pixel in the respective frame a likelihood of belonging to at least one surgical instrument to generate a respective mask for each surgical instrument in the respective frame;
• processing each of the masks within the respective frames with the plurality of parameters representative of the decomposition model to generate key point mask characteristics based on extracting surgical instrument characteristics for each mask in the respective frame;
• processing the key point mask characteristics in each frame with the plurality of parameters representative of the sequential relation architecture to generate the value related to surgical instrument motion;
• updating the plurality of parameters based on a loss value, the loss value based on a comparison of the respective label and the value related to surgical instrument motion;
• and
• storing the trained machine learning architecture in a memory.