Systems and methods for ai-assisted medical image annotation

A system comprising:

• one or more processors;
• a memory coupled to the one or more processors, the memory storing computer-program instructions, that, when executed by the one or more processors, perform a computer-implemented method comprising:
• extracting feature points from a single still image of the medical procedure and each frame of a sequential group of frames from a video of the medical procedure using a first machine learning algorithm, the still image and the video generated by or derived from an imaging device introduced into an interior of the patient during the procedure;
• comparing the feature points of the still image to the feature points of each frame of the group of frames from the video so as to determine a match between the still image and a given frame from the video and a sequence location in the video where the matched still image occurred, the comparison being done using a second machine learning algorithm, wherein a sequence number of a particular still image is used to reduce a size of the group of frames from which image features are extracted or compared to the still image;
• copying a portion of the video containing the matched single image so as to create a video clip, the video clip having a selected duration before and/or after a sequential location of the still image in the video;
• analyzing a plurality of frames in the video clip so as to identify image features in the video clip, wherein a fidelity of identification of a particular image feature is substantially unaffected when the image feature is non localizable or at least partially obscured, the analysis performed using a third machine learning algorithm;
• and
• annotating at least one of the still image or the video clip with information from the medical procedure, the information including or being derived from the at least one image feature.

A non-transitory computer-readable medium including contents that are configured to cause one or more processors to perform a method comprising:

• extracting feature points from a single still image of the medical procedure and each frame of a sequential group of frames from a video of the medical procedure using a first machine learning algorithm, the still image and the video generated by or derived from an imaging device introduced into an interior of the patient during the procedure;
• comparing the feature points of the still image to the feature points of each frame of the group of frames from the video so as to determine a match between the still image and a given frame from the video and a sequence location in the video where the matched still image occurred, the comparison being done using a second machine learning algorithm, wherein a sequence number of a particular still image is used to reduce a size of the group of frames from which image features are extracted or compared to the still image;
• copying a portion of the video containing the matched single image so as to create a video clip, the video clip having a selected duration before and/or after a sequential location of the still image in the video;
• analyzing a plurality of frames in the video clip so as to identify image features in the video clip, wherein a fidelity of identification of a particular image feature is substantially unaffected when the image feature is non localizable or at least partially obscured, the analysis performed using a third machine learning algorithm;
• and
• annotating at least one of the still image or the video clip with information from the medical procedure, the information including or being derived from the at least one image feature.