Systems, methods, and/or media, for selecting candidates for annotation for use in training a classifier

A method for selecting and labeling a Region of Interest (ROI) in an original medical image by fine-tuning a pre-trained convolutional neural network (CNN) using annotated medical images, wherein the method comprises:

• receiving the annotated medical images for use in fine-tuning the pre-trained CNN;
• generating a plurality of input candidates for fine-tuning the pre-trained CNN from the annotated medical images by performing one or more data augmentation operations, each of the plurality of input candidates for use in fine-tuning the pre-trained CNN having multiple identically labeled patches derived from the annotated medical images;
• providing the plurality of input candidates having the plurality of identically labeled patches as input to the pre-trained CNN;
• executing the pre-trained CNN to determine a plurality of probabilities for each of the plurality of input candidates, wherein each of the plurality of probabilities define a likelihood that a unique patch amongst the plurality of identically labeled patches of the respective input candidate corresponds to a label associated with annotations for the annotated medical images from which the respective input candidate was derived;
• determining an entropy value for each of the plurality of input candidates based of the plurality of probabilities of the corresponding input candidate;
• sorting the plurality of input candidates ding to their respective entropy value;
• identifying a subset of top-ranked input candidates amongst the sorted plurality of input candidates, wherein the subset does not include all of the plurality of input candidates;
• labeling the subset of top-ranked input candidates to produce labeled candidates;
• fine-tuning the pre-trained CNN using the labeled candidates to produce a trained CNN;
• and
• selecting and labeling the ROI within the original medical image and identifying via the trained CNN a location for the ROI within the original medical image using an ROI object confidence map produced by the trained CNN, wherein the location of the ROI is determined by identifying a largest connected object in the original medical image which is connected to an identified object within the ROI object confidence map based on at least one pixel of an object in the original medical image falling within the ROI;
• and
• returning as output, the selected and labeled ROI for the original medical image.

Non-transitory computer readable storage media having instructions stored thereupon that, when executed by a processor and a memory of a system, the instructions cause the system to perform operations comprising:

• receiving annotated medical images for use in fine-tuning a pre-trained CNN;
• generating a plurality of input candidates for fine-tuning the pre-trained CNN from the annotated medical images by performing one or more data augmentation operations, each of the plurality of input candidates for use in fine-tuning the pre-trained CNN having multiple identically labeled patches derived from the annotated medical images;
• providing the plurality of input candidates having the plurality of identically labeled patches as input to the pre-trained CNN;
• executing the pre-trained CNN to determine a plurality of probabilities for each of the plurality of input candidates, wherein each of the plurality of probabilities define a likelihood that a unique patch amongst the plurality of identically labeled patches of the respective input candidate corresponds to a label associated with annotations for the annotated medical images from which the respective input candidate was derived;
• determining an entropy value for each of the plurality of input candidates based on the plurality of probabilities of the corresponding input candidate;
• sorting the plurality of input candidates according to their respective entropy value;
• identifying a subset of top-ranked input candidates amongst the sorted plurality of input candidates, wherein the subset does not include all of the plurality of input candidates;
• labeling the subset of top-ranked input candidates to produce labeled candidates;
• fine-tuning the pre-trained CNN using the labeled candidates to produce a trained CNN;
• and
• selecting and labeling a region of interest (ROI) within the annotated medical images and identifying via the trained CNN a location for the ROI within the annotated medical images using an ROI object confidence map produced by the trained CNN, wherein the location of the ROI is determined by identifying a largest connected object in the annotated medical images which is connected to an identified object within the ROI object confidence map based on at least one pixel of an object in the annotated medical images falling within the ROI;
• and
• returning as output, the selected and labeled ROI for the annotated medical images.