Using Gradients to Detect Backdoors in Neural Networks

A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to implement a backdoor trigger identification engine, for evaluating a trained machine learning model to determine whether the machine learning model has a backdoor trigger, wherein the data processing system operates to:

• process, by the trained machine learning model, a test dataset to generate output classifications for the test dataset;
• generate, by the backdoor trigger identification engine, for the test dataset, gradient data indicating a degree of change of elements within the test dataset based on the output generated by processing the test dataset;
• analyze, by the backdoor trigger identification engine, the gradient data to identify a pattern of elements within the test dataset indicative of a backdoor trigger;
• and
• generate, by the backdoor trigger identification engine, in response to the analysis identifying the pattern of elements indicative of a backdoor trigger, an output indicating the existence of the backdoor trigger in the trained machine learning model.

An apparatus comprising:

• at least one processor;
• and
• at least one memory coupled to the at least one processor, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement a backdoor trigger identification engine, for evaluating a trained machine learning model to determine whether the machine learning model has a backdoor trigger, wherein the processor operates to:
• process, by the trained machine learning model, a test dataset to generate output classifications for the test dataset;
• generate, by the backdoor trigger identification engine, for the test dataset, gradient data indicating a degree of change of elements within the test dataset based on the output generated by processing the test dataset;
• analyze, by the backdoor trigger identification engine, the gradient data to identify a pattern of elements within the test dataset indicative of a backdoor trigger;
• and
• generate, by the backdoor trigger identification engine, in response to the analysis identifying the pattern of elements indicative of a backdoor trigger, an output indicating the existence of the backdoor trigger in the trained machine learning model.