Robustness to adversarial behavior for text classification models

A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform a method of training a machine learning model having a plurality of parameters and configured to generate a classification of a text sample comprising a plurality of words by processing a model input that includes a combined feature representation of the plurality of words in the text sample, wherein the combined feature representation is a representation in an embedding space,

• wherein the training comprises:
• receiving a training text sample and a target classification for the training text sample;
• generating a plurality of perturbed combined feature representations, wherein each perturbed combined feature representation is a representation in the embedding space, and wherein generating each of the perturbed combined feature representations comprises:
• selecting one or more words in the training text sample;
• generating a perturbed training text sample by determining a respective perturbed word for each selected word;
• and
• generating a combined feature representation of the perturbed training text sample;
• determining, based on the plurality of perturbed combined feature representations, a region in the embedding space;
• and
• determining an update to the parameters based on an adversarial objective that encourages the machine learning model to assign the target classification for the training text sample to all of the combined feature representations in the region in the embedding space.

One or more non-transitory computer storage media encoded with computer program instructions that when executed by a plurality of computers cause the plurality of computers to perform a method of training a machine learning model having a plurality of parameters and configured to generate a classification of a text sample comprising a plurality of words by processing a model input that includes a combined feature representation of the plurality of words in the text sample, wherein the combined feature representation is a representation in an embedding space,

• wherein the training comprises:
• receiving a training text sample and a target classification for the training text sample;
• generating a plurality of perturbed combined feature representations, wherein each perturbed combined feature representation is a representation in the embedding space, and wherein generating each of the perturbed combined feature representations comprises:
• selecting one or more words in the training text sample;
• generating a perturbed training text sample by determining a respective perturbed word for each selected word;
• and
• generating a combined feature representation of the perturbed training text sample;
• determining, based on the plurality of perturbed combined feature representations, a region in the embedding space;
• and
• determining an update to the parameters based on an adversarial objective that encourages the machine learning model to assign the target classification for the training text sample to all of the combined feature representations in the region in the embedding space.