Determination of the image components relevant to decision-making for an image classifier through modification in latent space

A method for measuring components of an input image on which an image classifier bases its decision regarding an assignment of the input image to one or multiple classes of a predefined classification, the method comprising the following steps:

• processing the input image by the image classifier into an intermediate product using one or multiple convolutional layers;
• mapping the intermediate product by the image classifier on a classification score with respect to at least one target class;
• ascertaining a perturbation in a space of the intermediate product from one or multiple counter images which is preferentially assigned by the image classifier to at least one class other than the target class;
• providing at least one binary mask, which has the same number of pixels as the intermediate product;
• creating, from the intermediate product, at least one modification, in which pixels of the intermediate product established by the binary mask are replaced with pixels of the perturbation corresponding the pixels established by the binary marked;
• mapping the modification by the image classifier on a classification score with respect to a predefined class;
• and
• ascertaining from the classification score, using a quality function, to what extent the binary mask indicates sought-after decision-relevant components of the input image.

A non-transitory machine-readable data medium on which is stored a computer program including machine-readable instructions for measuring components of an input image on which an image classifier bases its decision regarding an assignment of the input image to one or multiple classes of a predefined classification, the machine-readable instructions, when executed by one or more computers, causing the one or more computers to perform the following steps:

• processing the input image by the image classifier into an intermediate product using one or multiple convolutional layers;
• mapping the intermediate product by the image classifier on a classification score with respect to at least one target class;
• ascertaining a perturbation in a space of the intermediate product from one or multiple counter images which is preferentially assigned by the image classifier to at least one class other than the target class;
• providing at least one binary mask, which has the same number of pixels as the intermediate product;
• creating, from the intermediate product, at least one modification, in which pixels of the intermediate product established by the binary mask are replaced with pixels of the perturbation corresponding the pixels established by the binary marked;
• mapping the modification by the image classifier on a classification score with respect to a predefined class;
• and
• ascertaining from the classification score, using a quality function, to what extent the binary mask indicates sought-after decision-relevant components of the input image.

A computer configured to measure components of an input image on which an image classifier bases its decision regarding an assignment of the input image to one or multiple classes of a predefined classification, the computer configured to:

• process the input image by the image classifier into an intermediate product using one or multiple convolutional layers;
• map the intermediate product by the image classifier on a classification score with respect to at least one target class;
• ascertain a perturbation in a space of the intermediate product from one or multiple counter images which is preferentially assigned by the image classifier to at least one class other than the target class;
• provide at least one binary mask, which has the same number of pixels as the intermediate product;
• create, from the intermediate product, at least one modification, in which pixels of the intermediate product established by the binary mask are replaced with pixels of the perturbation corresponding the pixels established by the binary marked;
• map the modification by the image classifier on a classification score with respect to a predefined class;
• and
• ascertain from the classification score, using a quality function, to what extent the binary mask indicates sought-after decision-relevant components of the input image.