Dominant tool detection system for surgical videos

A system comprising:

• one or more processors;
• and
• logic encoded in one or more non-transitory computer-readable storage media for execution by the one or more processors and when executed operable to perform operations comprising:
• receiving at least one image frame;
• detecting one or more objects in the at least one image frame;
• classifying the one or more objects into one or more tool classifications, wherein the one or more objects are one or more tools;
• determining if each tool of the one or more tools is non-assistive based on the one or more tool classifications;
• determining a handedness of the one or more tools;
• and
• determining a dominant tool from the one or more tools based at least in part on the dominant tool being determined as non-assistive and based at least in part on the handedness of the one or more tools.