Virtual pointer for real-time endoscopic video using gesture and voice commands and video architecture and framework for collecting surgical video at scale

A method of using medical software tools system, comprising:

• a tool for sensing surgeon hand movements in connection with surgical camera usage to invoke a video overlay displaying a synthetic visual path starting from an end of a selected instrument onward through an intended path of movement in a specified distance and direction corresponding to an orientation of said tool;
• a computer system receiving an image stream from said surgical camera;
• said computer system providing a user interface overlay adapted for presentation over a surgical image stream and analyzing said surgical image steam and calculating an anticipated direction of movement corresponding to said direction said surgical tool is oriented;
• and
• use of a combination of audio keywords and movements to enable predictive surgical tool movements to assist said surgeon.

A medical software tools system, comprising:

• a tool for sensing surgeon hand movements in connection with surgical camera usage to invoke a video overlay displaying a synthetic visual path starting from an end of a selected instrument onward through an intended path of movement in a specified distance and direction corresponding to an orientation of said tool;
• a computer system receiving an image stream from said surgical camera;
• said computer system providing a user interface overlay adapted for presentation over an surgical image stream and analyzing said surgical image steam and calculating an anticipated direction of movement corresponding to said direction said surgical tool is oriented;
• use of a combination of audio keywords and movements to enable predictive surgical tool movements to assist said surgeon;
• and
• a cloud network collects surgical images on an enterprise scale to enable hospitals to ingest, manage, and fully utilize patient surgical video within a hospital network and to share said video with designated users and to automatically record video during surgery and provide key clips for evaluation of hospital resources and to store said video for integration with patient electronic health records.