Automatic detection of mitosis using handcrafted and convolutional neural network features

A non-transitory computer-readable storage medium storing computer-executable instructions that when executed by a computer control the computer to perform a method for producing a classification of a region of interest, the method comprising:

• accessing an image of the region of interest;
• generating a first classification for the image using a first classifier trained using a first feature set, where the first classifier is a light convolutional neural network (CNN);
• generating a second classification for the image using a second classifier trained using a second feature set, where the second feature set is distinguishable from the first feature set, and where the second feature set is a hand-crafted (HC) feature set;
• upon determining that the first classification and the second classification are within a threshold amount, producing a final classification for the image based, at least in part, on the first classification and the second classification;
• upon determining that the first classification and the second classification are not within the threshold amount, generating a third classification for the image using a third classifier trained using the first feature set and the second feature set, and producing the final classification for the image based, at least in part, on a weighted average of the first classification, the second classification, and the third classification, and
• controlling an automated image classification system to classify the image based on the final classification.

An apparatus for detecting mitosis in cancer pathology images, comprising:

• a processor;
• a memory;
• an input/output interface;
• a set of logics;
• and
• an interface to connect the processor, the memory, the input/output interface and the set of logics, the set of logics comprising:
• an image acquisition logic that acquires an image of a region of tissue;
• a convolutional neural network (CNN) logic that extracts a set of CNN-learned features from a patch of the image and generates a first probability that the patch is mitotic, based, at least in part, on the set of CNN-learned features, where the patch of the image is smaller than the image;
• a hand-crafted (HC) features logic that extracts a set of HC features from the patch and generates a second probability that the patch is mitotic, based, at least in part, on the set of HC features;
• a first classification logic that classifies the patch as mitotic when the first probability and the second probability are both greater than a threshold probability, classifies the patch as non-mitotic when the first probability and the second probability are both less than or equal to the threshold probability, and classifies the patch as confounding when one of the first probability or the second probability is greater than the threshold probability and the other, different probability is less than or equal to the threshold probability;
• a cascaded architecture logic that generates a third probability that the patch is mitotic when the first classification logic classified the patch as confounding, and generates the third probability that the patch is mitotic based, at least in part, on a stacked set of features, where the stacked set of features includes the set of CNN-learned features and the set of HC features, and
• a second classification logic that generates a fourth probability that the patch is mitotic based, at least in part, on a weighted average of the first probability, the second probability, and the third probability, where the second classification logic classifies the patch as mitotic when the weighted average is greater than the threshold probability, and classifies the patch as non-mitotic when the weighted average is less than or equal to the threshold probability.