Methods and apparatus to improve deepfake detection with explainability

A non-transitory computer readable storage medium comprising instructions to cause at least one programmable circuit to at least:

• train a classification model based on a first portion of a dataset of media files with known classification information, the classification model to output a classification for input media file from a second portion of the dataset of media files with the known classification information;
• compare the classification of the input media file from the classification model with a known classification of the input media file to determine if a misclassification occurred;
• when the input media file is misclassified, determine a reason for the misclassification based on a comparison of first patterns of an explainability map of the misclassified input media file to second patterns of correctly classified explainability maps, the explainability map to identify a part within the input media file responsible for generating the classification of the input media file;
• modify the classification model based on the comparison of the first patterns to the second patterns;
• and
• generate the explainability map based on an average weighted feature map along channel dimensions of a feature map.

A method comprising:

• training, by executing an instruction with programmable circuitry, a classification model based on a first portion of a dataset of media files with known classification information, the classification model to output a classification for input media file from a second portion of the dataset of media files with the known classification information;
• comparing, by executing an instruction with the programmable circuitry, the classification of the input media file from the classification model with a known classification of the input media file to determine if a misclassification occurred;
• when the input media file is misclassified, determining, by executing an instruction with the programmable circuitry, a reason for the misclassification based on a comparison of first patterns of an explainability map of the misclassified input media file to second patterns of correctly classified explainability maps, the explainability map to identify a part within the input media file responsible for generating the classification of the input media file;
• modifying, by executing an instruction with the programmable circuitry, the classification model based on the comparison of the first patterns to the second patterns;
• and
• calculating, by executing an instruction with the programmable circuitry, an average gradient value, the average gradient value used to weigh a channel of a feature map to generate the explainability map.

An apparatus comprising:

• interface circuitry;
• instructions;
• and
• at least one programmable circuit to be programmed by the instructions to:
• train a classification model based on a first portion of a dataset of media files with known classification information, the classification model to output a classification for input media file from a second portion of the dataset of media files with the known classification information;
• compare the classification of the input media file from the classification model with a known classification of the input media file to determine if a misclassification occurred;
• when the input media file is misclassified, determine a reason for the misclassification based on a comparison of first patterns of an explainability map of the misclassified input media file to second patterns of correctly classified explainability maps, the explainability map to identify a part within the input media file responsible for generating the classification of the input media file;
• modify the classification model based on the reason for the misclassification;
• and
• generate the explainability map based on an average weighted feature map along channel dimensions of the feature map.