Privacy preserving machine learning model training

One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training of a machine learning model having a plurality of model parameters on user data from a plurality of users while preserving privacy of the user data, the operations comprising:

• initializing a respective mean noisy gradient estimate for each of the plurality of model parameters;
• initializing a respective standard deviation noisy gradient estimate for each of the plurality of model parameters;
• and
• at each of a plurality of training steps:
• obtaining, over a data communication network, a plurality of training examples for the training step comprising data from a plurality of users;
• and
• training the machine learning model using the plurality of training examples for the training step, the training comprising:
• (i) updating the model parameters while preserving privacy of user data within the plurality of training examples using the respective mean noisy gradient estimate and the respective standard deviation noisy gradient estimate for each of the model parameters, and
• (ii) updating the respective mean noisy gradient estimate and the respective standard deviation noisy gradient estimate for each of the model parameters using the plurality of training examples, wherein updating the model parameters using the plurality of training examples comprises:
• for each of the plurality of training examples:
• generating, for the training example and for each of the plurality of model parameters, a respective gradient of an objective function for the training example with respect to the model parameter;
• and
• generating, for each of the plurality of model parameters, a respective privacy preserving noisy gradient for the training example that preserves privacy of user data contained within the training example by modifying the respective gradient for the model parameter for the training example based on (i) the mean noisy gradient estimate for the model parameter after being updated using privacy preserving noisy gradients with respect to the model parameter determined at a preceding training step and (ii) the standard deviation noisy gradient estimate for the model parameter after being updated using the mean noisy gradient estimate and the privacy preserving noisy gradients determined with respect to model parameter at the preceding training step, and
• wherein generating the respective privacy preserving noisy gradient comprises, for each model parameter:
• transforming the gradient for the model parameter based on (i) the mean noisy gradient estimate for the model parameter and (ii) the standard deviation noisy gradient estimates for the plurality of model parameters to generate a transformed gradient;
• and
• generating an adaptively clipped gradient, comprising clipping, to have a fixed norm, the transformed gradient for the model parameter that has been transformed using the respective mean and standard deviation noisy gradient estimates for the model parameter;
• generating, for each of the model parameters, a respective privacy preserving update for the model parameter for the training examples for the training step that preserves privacy of user data contained within the training examples for the training step from the privacy preserving noisy gradients for the model parameter for the plurality of training examples;
• and
• updating the model parameters with the privacy preserving updates;
• and wherein updating the respective mean noisy gradient estimate and the respective standard deviation noisy gradient estimate for each of the model parameters comprises, for each of the plurality of model parameters:
• updating, for use at a subsequent training step, the standard deviation noisy gradient estimate for the model parameter based on the mean estimate for the model parameter and the privacy preserving noisy gradients for the model parameter;
• and
• updating, for use at the subsequent training step, the mean noisy gradient estimate for the model parameter based on the privacy preserving noisy gradients for the model parameter.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for training of a machine learning model having a plurality of model parameters on user data from a plurality of users while preserving privacy of the user data, the operations comprising:

• initializing a respective mean noisy gradient estimate for each of the plurality of model parameters;
• initializing a respective standard deviation noisy gradient estimate for each of the plurality of model parameters;
• and
• at each of a plurality of training steps:
• obtaining, over a data communication network, a plurality of training examples for the training step comprising data from a plurality of users;
• and
• training the machine learning model using the plurality of training examples for the training step, the training comprising:
• (i) updating the model parameters while preserving privacy of user data within the plurality of training examples using the respective mean noisy gradient estimate and the respective standard deviation noisy gradient estimate for each of the model parameters, and
• (ii) updating the respective mean noisy gradient estimate and the respective standard deviation noisy gradient estimate for each of the model parameters using the plurality of training examples, wherein updating the model parameters using the plurality of training examples comprises:
• for each of the plurality of training examples:
• generating, for the training example and for each of the plurality of model parameters, a respective gradient of an objective function for the training example with respect to the model parameter;
• and
• generating, for each of the plurality of model parameters, a respective privacy preserving noisy gradient for the training example that preserves privacy of user data contained within the training example by modifying the respective gradient for the model parameter for the training example based on (i) the mean noisy gradient estimate for the model parameter after being updated using privacy preserving noisy gradients with respect to the model parameter determined at a preceding training step and (ii) the standard deviation noisy gradient estimate for the model parameter after being updated using the mean noisy gradient estimate and the privacy preserving noisy gradients determined with respect to model parameter at the preceding training step, and
• wherein generating the respective privacy preserving noisy gradient comprises, for each model parameter:
• transforming the gradient for the model parameter based on (i) the mean noisy gradient estimate for the model parameter and (ii) the standard deviation noisy gradient estimates for the plurality of model parameters to generate a transformed gradient;
• and
• generating an adaptively clipped gradient, comprising clipping, to have a fixed norm, the transformed gradient for the model parameter that has been transformed using the respective mean and standard deviation noisy gradient estimates for the model parameter;
• generating, for each of the model parameters, a respective privacy preserving update for the model parameter for the training examples for the training step that preserves privacy of user data contained within the training examples for the training step from the privacy preserving noisy gradients for the model parameter for the plurality of training examples;
• and
• updating the model parameters with the privacy preserving updates;
• and wherein updating the respective mean noisy gradient estimate and the respective standard deviation noisy gradient estimate for each of the model parameters comprises, for each of the plurality of model parameters:
• updating, for use at a subsequent training step, the standard deviation noisy gradient estimate for the model parameter based on the mean estimate for the model parameter and the privacy preserving noisy gradients for the model parameter;
• and
• updating, for use at the subsequent training step, the mean noisy gradient estimate for the model parameter based on the privacy preserving noisy gradients for the model parameter.