Systems and methods for explaining machine learning models

A method for explaining a decision process of a machine learning (ML) model, the method comprising:

• obtaining a prediction about an image, wherein the prediction is made based on a first ML model;
• determining visual concepts associated with the image that are used by the first ML model to make the prediction;
• determining respective contributions of the visual concepts to the prediction made by the first ML model;
• and
• generating, based on a second ML model, a textual description that explains the respective contributions of the visual concepts to the prediction made by the first ML model;
• wherein the second ML model includes a graph reasoning neural network pre-trained to generate a graph representation that indicates a reasoning process of the first ML model when making the prediction about the image, the graph representation comprising nodes and edges, each node representing a respective one of the visual concepts, each edge connecting two nodes and representing a dependency of the respective visual concepts represented by the two nodes during the reasoning process of the first ML model, and wherein the second ML model further includes a vision-language model pre-trained to determine respective image features associated with the visual concepts, map the determined image features to corresponding text features, and generate the textual description based at least on the text features.