System and method for identifying misclassifications by a neural network

A system for identifying misclassification in a neural network, the system comprising:

• a processor;
• and
• a memory communicatively coupled to the processor, wherein the memory stores processor instructions, which, on execution, causes the processor to:
• generate for an input image, an image heatmap associated with each of a plurality of layers of a neural network, wherein the input image is associated with a class of a plurality of classes;
• determine for the input image, an activation value at each of the plurality of layers based on the associated image heatmap, wherein the activation value for a layer of the plurality of layers corresponds to a weighted sum of a plurality of neurons in the layer;
• identify for the input image, a first pattern of triggered neurons in at least one of the plurality of layers based on the activation value generated for the at least one of the plurality of layers;
• compare for each of the plurality of classes, a second pattern of triggered neurons with the first pattern of triggered neurons identified at the at least one of the plurality of layers, wherein, for each of the plurality of classes, the second pattern of triggered neurons is identified for a penultimate layer of the plurality of layers, based on an activation value generated at the penultimate layer;
• and
• identify a misclassification of the input image in the neural network based on a result of the comparison.

A non-transitory computer-readable storage medium having stored thereon, a set of computer-executable instructions causing a computer comprising one or more processors to perform steps comprising:

• generating for an input image, an image heatmap associated with each of a plurality of layers of a neural network, wherein the input image is associated with a class of a plurality of classes;
• determining for the input image, an activation value at each of the plurality of layers based on the associated image heatmap, wherein the activation value for a layer of the plurality of layers corresponds to a weighted sum of a plurality of neurons in the layer;
• identifying for the input image, a first pattern of triggered neurons in at least one of the plurality of layers based on the activation value generated for the at least one of the plurality of layers;
• comparing for each of the plurality of classes, a second pattern of triggered neurons with the first pattern of triggered neurons identified at the at least one of the plurality of layers, wherein, for each of the plurality of classes, the second pattern of triggered neurons is identified for a penultimate layer of the plurality of layers, based on an activation value generated at the penultimate layer;
• and
• identifying a misclassification of the input image in the neural network based on a result of the comparison.