Saliency mapping by feature reduction and perturbation modeling in medical imaging

A method for machine training saliency map prediction of a machine-learned classifier, the method comprising:

• classifying an input image by the machine-learned classifier;
• back-propagating from a class of the classifying, the back-propagating providing gradients with respect to feature maps of the machine-learned classifier;
• selecting less than all of the feature maps based on sensitivities of the feature maps to the class;
• generating a perturbation mask by input of the selected features maps to a perturbation encoder;
• inputting a perturbed input formed by combining the perturbation mask with the input image to the machine-learned classifier, the inputting resulting in a distorted class prediction;
• computing a score from the distorted class prediction;
• and
• altering the perturbation encoder to minimize the score.

A system for saliency map prediction, the system comprising:

• a medical imaging scanner configured to generate data representing a patient;
• an image processor configured to classify the data by application of a machine-learned model and to generate a saliency map for the classification by application of input based on gradients from back-propagation of the classification to a perturbation encoder, the perturbation encoder having been machine trained in a semi-supervised manner based on entropy and a difference of saliency from ground truth segmentations;
• and
• a display configured to display the classification and the saliency map for the patient.