Methods and systems for improved ai model development for medical applications

A medical image analysis system, comprising:

• an image database comprising a plurality of medical images, wherein a minority of the plurality of medical images are annotated by one or more experts and wherein a majority of the plurality of medical images are not annotated, an annotation comprising an identification/classification of one or more features in a medical image;
• a trained medical image analysis algorithm configured to identify/classify the one or more features in a medical image and to generate an uncertainty score for a medical image, the uncertainty score comprising a quantification of confidence of the trained medical image analysis algorithm identify/classify the one or more features in the medical image;
• and
• a processor configured to perform one or more iterations of:
• analyzing, using the trained medical image analysis algorithm, the plurality of medical images that are not annotated to generate an uncertainty score for at least some of the plurality of medical images;
• (ii) prioritizing for expert annotation, based on the generated uncertainty scores, a subset of the plurality of medical images that are not annotated;
• (iii) receiving, after expert annotation, the subset of the plurality of medical images;
• and (iv) retraining, using at least the received annotated subset of the plurality of medical images, the medical image analysis algorithm, wherein an updated trained medical image analysis algorithm is generated upon completion of the retraining.

A non-transitory, computer readable medium, comprising instructions that in response to execution on a processor, cause the processor to:

• receive a collection of medical images, wherein a portion of the collection is annotated by at least one expert and a remaining portion is unannotated, each annotation comprising identification or classification of one or more features within a medical image;
• executing at least one cycle comprising:
• analyzing, by a trained medical image analysis algorithm, at least some of the unannotated medical images to generate uncertainty scores, wherein the medical image analysis algorithm training used at least the annotated portion of the collection to enable identification or classification of the one or more features and to generate an uncertainty score for a medical image, the uncertainty score representing a confidence level of the identification or classification by the algorithm;
• selecting, based on the uncertainty scores, a subset of the unannotated medical images for expert annotation;
• receiving expert annotations for the selected subset;
• and
• retraining the medical image analysis algorithm using at least the newly annotated subset, thereby producing an updated medical image analysis algorithm;
• wherein the updated medical image analysis algorithm is configured to identify or classify features in medical images and to generate uncertainty scores, and wherein the iterative training continues until a predetermined performance criterion is met.