Training more secure neural networks by using local linearity regularization

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:

• receiving a network input;
• and
• processing the network input using a neural network and in accordance with trained values of a plurality of network parameters of the neural network to generate a network output for the network input, wherein:
• the trained values of the plurality of network parameters have been determined at least in part by training the neural network on a plurality of training inputs and corresponding target outputs to (i) decrease respective losses for the training inputs that are computed by, for each training input, evaluating a loss function at an input-output pair that includes the training input and the corresponding target output for the training input and (ii) decrease a non-linearity of the loss function for a maximally non-linear perturbation, and
• the maximally non-linear perturbation is a perturbation from a set of possible perturbations for which the loss function is most non-linear when evaluated at an input-output pair that includes (i) a perturbed training input generated by applying the possible perturbation to a given training input and (ii) a target output for the given training input.

One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• receiving a network input;
• and
• processing the network input using a neural network and in accordance with trained values of a plurality of network parameters of the neural network to generate a network output for the network input, wherein:
• the trained values of the plurality of network parameters have been determined at least in part by training the neural network on a plurality of training inputs and corresponding target outputs to (i) decrease respective losses for the training inputs that are computed by, for each training input, evaluating a loss function at an input-output pair that includes the training input and the corresponding target output for the training input and (ii) decrease a non-linearity of the loss function for a maximally non-linear perturbation, and
• the maximally non-linear perturbation is a perturbation from a set of possible perturbations for which the loss function is most non-linear when evaluated at an input-output pair that includes (i) a perturbed training input generated by applying the possible perturbation to a given training input and (ii) a target output for the given training input.