Detecting backdoor attacks using exclusionary reclassification

A computer program product to utilize machine learning to process an untrusted training data set, the computer program product comprising:

• a computer readable storage medium having program code embodied therewith, the program code executable by a processor to:
• train a first neural model with the untrusted data set;
• classify each data point in the untrusted data set using the trained first neural model, and retain activations of one or more designated layers in the trained first neural model;
• apply a clustering technique on the retained activations for each label, and for each cluster assess integrity of data in the cluster, including program code to:
• remove a cluster identified as containing suspect data from the data set, and form a new training set with data remaining in the data set without the suspect data;
• train a second neural model using the new training set;
• and
• using the trained second neural model, analyze data in the removed cluster and assess alignment of one or more of the classified data points with respect to a label assignment;
• and
• assign a poisonous classification or a legitimate classification to the removed cluster, the cluster classification corresponding to the alignment assessment.

A method comprising:

• receiving, by a neural network, an untrusted training data set, each data point of the untrusted data set having a label;
• training a first neural model using the untrusted data set;
• classifying each data point in the untrusted data set using the trained first neural model, and retaining activations of one or more designated layers in the trained first neural model;
• applying a clustering technique on the retained activations for each label;
• for each cluster, assessing integrity of data in the cluster including:
• removing a cluster identified as containing suspect data from the data set, and forming a new training set with data remaining in the data set without the suspect data;
• training a second neural model using the new training set;
• and
• using the trained second neural model, analyzing data in the removed cluster and assessing alignment of one or more of the classified data points with respect to a label assignment;
• and
• assigning poisonous classification or a legitimate a classification to the removed cluster, the cluster classification corresponding to the alignment assessment.