Training User-Level Differentially Private Machine-Learned Models

A computing system, comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media that collectively store instructions that are executable by the one or more processors to cause the computing system to perform operations, the operations comprising:
• receiving, from one or more client computing devices from a pool of available client computing devices, a local update for a machine-learned model;
• determining a differentially private aggregate of the local updates, wherein the differentially private aggregate satisfies a differential privacy metric such that a sensitivity of the aggregate to data from any one client is bounded;
• determining an updated machine-learned model based at least in part on the differentially private aggregate of the local updates;
• and
• transmitting, through a network interface, one or more model parameters of the updated machine-learned model for delivery to one or more of the available client computing devices, wherein a sensitivity of the transmitted one or more model parameters to the data from any one client is bounded.

One or more non-transitory computer-readable media that collectively store instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising:

• receiving, from one or more client computing devices from a pool of available client computing devices, a local update for a machine-learned model;
• determining a differentially private aggregate of the local updates, wherein the differentially private aggregate satisfies a differential privacy metric such that a sensitivity of the aggregate to data from any one client is bounded;
• determining an updated machine-learned model based at least in part on the differentially private aggregate of the local updates;
• and
• transmitting, through a network interface, one or more model parameters of the updated machine-learned model for delivery to one or more of the available client computing devices, wherein a sensitivity of the transmitted one or more model parameters to the data from any one client is bounded.

A method comprising:

• receiving, by a computing system comprising one or more processors, from one or more client computing devices from a pool of available client computing devices, a local update for a machine-learned model;
• determining, by the computing system, a differentially private aggregate of the local updates, wherein the differentially private aggregate satisfies a differential privacy metric such that a sensitivity of the aggregate to data from any one client is bounded;
• determining, by the computing system, an updated machine-learned model based at least in part on the differentially private aggregate of the local updates;
• and
• transmitting, by the computing system and through a network interface, one or more model parameters of the updated machine-learned model for delivery to one or more of the available client computing devices, wherein a sensitivity of the transmitted one or more model parameters to the data from any one client is bounded.