Adversarial training to minimize data poisoning attacks

A system for training a machine learning model to classify inputs comprising:

• one or more processing units;
• and
• a computer-readable memory having encoded thereon computer-readable instructions that when executed by the one or more processing units, causes the system to:
• receive an initial dataset from a trusted source, the initial dataset comprising a first plurality of inputs, wherein an individual input of the first plurality of inputs includes a natural language input and a known label;
• train the machine learning model to classify each of the first plurality of inputs based on the natural language input and the known label;
• generate an augmented dataset by modifying one or more natural language inputs included in one or more of the first plurality of inputs of the initial dataset to include a known adversarial trigger;
• generate a robust machine learning model by training the machine learning model to identify one or more types of adversarial triggers utilizing the augmented dataset including the known adversarial trigger;
• receive an untrusted dataset from an external source, the untrusted dataset comprising a second plurality of inputs wherein one or more individual inputs of the second plurality of inputs comprises a compromised natural language input including an adversarial trigger;
• and
• retrain the robust machine learning model using the untrusted dataset and the augmented dataset to generate an updated robust machine learning model for detecting a subsequent compromised natural language input.

A method comprising:

• receiving an initial dataset from a trusted source, the initial dataset comprising a first plurality of inputs, wherein an individual input of the first plurality of inputs includes a natural language input and a known label;
• training the machine learning model to classify each of the first plurality of inputs based on the natural language input and the known label;
• generating an augmented dataset by modifying one or more natural language inputs included in one or more of the first plurality of inputs of the initial dataset to include a known adversarial trigger;
• generating a robust machine learning model by training the machine learning model to identify one or more types of adversarial triggers utilizing the augmented dataset including the known adversarial trigger;
• receiving an untrusted dataset from an external source, the untrusted dataset comprising a second plurality of inputs wherein one or more individual inputs of the second plurality of inputs comprises a compromised natural language input including an adversarial trigger;
• retraining the robust machine learning model using the untrusted dataset and the augmented dataset to generate an updated robust machine learning model for detecting a subsequent compromised natural language input;
• detecting an error at an output of the robust machine learning model;
• in response to detecting the error, providing a plurality of datasets that were used to train the robust machine learning model to a backdoor classifier comprising a first classifier and a second classifier;
• determining, at the first classifier, a subset of the datasets that caused the error;
• determining, at the second classifier, a type of compromise of the subset of the datasets that caused the error;
• and
• generating an auditing output indicating the subset of datasets that caused the error and the type of compromise.