Method and system for classification of endoscopic images using deep decision networks

A method of classifying endoscopic images, comprising:

• receiving a plurality of endoscopic images;
• classifying each of the plurality of endoscopic images and determining a confidence score for each of the plurality of endoscopic images using an initial trained deep network classifier;
• comparing the confidence score for each of the plurality of endoscopic images to a learned confidence threshold to determine a highly confident subset of the plurality of endoscopic images and a confusion subset of the plurality of endoscopic images;
• outputting classification results from the initial trained deep network classifier for the highly confident subset of the plurality of endoscopic images;
• and
• classifying each of the confusion subset of the plurality of endoscopic images using one or more specialized network classifiers,
• wherein classifying each of the confusion subset of the plurality of endoscopic images using one or more specialized network classifiers comprises:
• classifying of the endoscopic images in the confusion subset using a first specialized network classifier built on a feature space of the initial trained deep network classifier.

An apparatus for classifying endoscopic images, comprising:

• means for classifying endoscopic images and determining confidence scores for the endoscopic images using an initial trained deep network classifier;
• means for comparing the confidence score for each endoscopic image classified by the initial trained deep network classifier to a learned confidence threshold;
• and
• means for classifying endoscopic images with confidence scores lower than the learned confidence threshold with a first specialized network classifier built on a feature space of the initial trained deep network classifier
• wherein an input layer of the initial trained deep network classifier inputs raw pixel data of endoscopic images, and an input layer of the first specialized network classifier inputs a feature vector output by a hidden layer of the initial trained deep network classifier.

A non-transitory computer readable medium storing computer program instructions for classifying endoscopic images, the computer program instructions when executed by a processor perform operations comprising:

• classifying an endoscopic image and determining a confidence score for the endoscopic image using an initial trained deep network classifier;
• comparing the confidence score for the endoscopic image to a learned confidence threshold;
• in response to a determination that the confidence score for the endoscopic image is higher than the learned confidence threshold, outputting the classification of the endoscopic image by the initial trained deep network classifier;
• and
• in response to a determination that the confidence score for the endoscopic image is not higher than the learned confidence threshold, classifying the endoscopic image with a first specialized network classifier built on a feature space of the initial trained deep network classifier
• wherein an input layer of the second specialized network classifier inputs a feature vector output by a hidden layer of the first specialized network classifier.