Dynamic gradient aggregation for training neural networks

A computerized method for training a global model based on a plurality of clients, the computerized method comprising:

• providing, by a processor, the global model to a plurality of clients;
• receiving, by the processor, a plurality of gradients from the plurality of clients;
• determining, by the processor, a plurality of gradient quality metrics, at least one gradient quality metric for each individual gradient of the plurality of gradients, the at least one gradient quality metric indicating a degree to which the individual gradient can be used to improve the global model;
• calculating, by the processor, a plurality of weight factors using the determined plurality of gradient quality metrics, the plurality of weight factors including an individual weight factor for each individual gradient of the plurality of gradients;
• transforming, by the processor, the plurality of gradients into a plurality of weighted gradients based on the calculated plurality of weight factors;
• generating, by the processor, a global gradient based on the plurality of weighted gradients;
• and
• updating, by the processor, the global model based on the global gradient, wherein the updated global model, when applied to an individual data set, performs a task based on the individual data set and provides model output based on performing the task.

One or more non-transitory computer storage media having computer-executable instructions for applying a global model to a data set that, upon execution by a processor, cause the processor to at least:

• access the data set;
• apply the global model to the data set, whereby the global model performs a task on the accessed data set and provides model output based on performing the task;
• train the global model based on the data set and the provided model output;
• generate a gradient associated with the data set based on training the global model;
• provide the generated gradient to a source of the global model, wherein the source of the global model is configured to:
• include the generated gradient in a plurality of gradients associated with the global model;
• calculate a plurality of weight factors, the plurality of weight factors including an individual weight factor for each individual gradient of the plurality of gradients;
• and
• update the global model based on the plurality of gradients and plurality of weight factors;
• and
• receive an updated global model from the source of the global model.