Method and system for meaningful counterfactual explanations

A system for explaining an image classifier, the system comprising:

• a processor;
• and
• a non-transitory storage medium operatively connected to the processor, the non-transitory storage medium comprising computer readable instructions;
• the processor, upon executing the computer readable instructions, being configured for:
• a) receiving an initial image, the initial image having been wrongly classified by the image classifier;
• b) receiving an initial gradient of a function executed by the image classifier generated while classifying the initial image, the function being indicative of a probability for the initial image to belong to an initial class;
• c) converting the initial image into a latent vector, the latent vector being a representation of the initial image in a latent space;
• d) generating a plurality of perturbation vectors using the initial gradient of the function executed by the image classifier;
• e) combining the latent vector with each one of the plurality of perturbation vectors, thereby obtaining a plurality of modified vectors;
• f) for each one of the plurality of modified vectors, reconstructing a respective image, thereby obtaining a plurality of reconstructed images;
• g) transmitting the reconstructed images to the image classifier;
• h) for each one of the plurality of reconstructed images, receiving a respective updated gradient of the function executed by the image classifier;
• i) using the respective updated gradients, determining amongst the reconstructed images at least one given reconstructed image for which the respective updated gradient is indicative that a new class different from the initial class has been assigned by the image classifier;
• and
• j) outputting the at least one given reconstructed image.

A system for explaining an image classifier, the system comprising:

• an encoder for:
• receiving an initial image, the initial image having been wrongly classified by the image classifier;
• and
• converting the initial image into a latent vector, the latent vector being a representation of the initial image in a latent space;
• a vector generator:
• receiving an initial gradient of a function executed by the image classifier generated while classifying the initial image, the function being indicative of a probability for the initial image to belong to an initial class;
• generating a plurality of perturbation vectors using the initial gradient of the function executed by the image classifier;
• and
• combining the latent vector with each one of the plurality of perturbation vectors, thereby obtaining a plurality of modified vectors;
• and
• a decoder for:
• for each one of the plurality of modified vectors, reconstructing a respective image, thereby obtaining a plurality of reconstructed images;
• and
• transmitting the reconstructed images to the image classifier,
• wherein the vector generator is further configured for:
• for each one of the plurality of reconstructed images, receiving a respective updated gradient of the function executed by the image classifier;
• using the respective updated gradients, determining amongst the reconstructed images at least one given reconstructed image for which the respective updated gradient is indicative that a new class different from the initial class has been assigned by the image classifier;
• and
• outputting an identification of the at least one given reconstructed image.