Systems and methods for frame-based validation

A method for validating performance of a trained model configured to predict at least one of a plurality of tissue and/or cellular characteristics categories from a pathology image, the method comprising, by at least a processor:

• receiving, from a plurality of users, reference annotations for one or more frames in a set of frames across one or more pathology images, wherein each reference annotation describes at least one of a plurality of tissue and/or cellular characteristic categories for a frame in the set of frames, and wherein each frame in the set of frames includes a portion of a pathology image of the one or more pathology images;
• receiving model predicted annotations each describing at least one of the plurality of tissue and/or cellular characteristic categories for a frame of the set of frames, wherein the predicted annotations are generated by the trained model;
• and
• determining performance of the trained model by associating the model predicted annotations with the reference annotations across the one or more pathology images from the plurality of users.

A system for validating performance of a trained model configured to predict at least one of a plurality of tissue and/or cellular characteristics categories from a pathology image, the system comprising:

• at least one computer hardware processor;
• and
• at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:
• receiving, from a plurality of users, reference annotations for one or more frames in a set of frames across one or more pathology images, wherein each reference annotation describes at least one of a plurality of tissue and/or cellular characteristic categories for a frame in the set of frames, and wherein each frame in the set of frames includes a portion of a pathology image of the one or more pathology images;
• receiving model predicted annotations each describing at least one of the plurality of tissue and/or cellular characteristic categories for a frame of the set of frames, wherein the predicted annotations are generated by the trained model;
• determining performance of the trained model by associating the model predicted annotations with the reference annotations of the one or more pathology images from the plurality of users;
• and
• evaluating the performance of the trained model against one or more criteria comprising:
• non-inferiority, equivalence, and/or superiority, to determine acceptability of the trained model.