# US 2022/0036178 A1 · Dynamic gradient aggregation for training neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036178 A1
- Document ID: US20220036178A1
- Status: Published application
- Kind: A1
- Application number: 16/945,715
- Filed: 2020-07-31
- Published: 2022-02-03
- Earliest claimed: 2020-07-31
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Dimitrios B. DIMITRIADIS; Kenichi KUMATANI; Robert Peter GMYR; Masaki ITAGAKI; Yashesh GAUR; Nanshan ZENG; Xuedong HUANG
- Web page: https://patentgrep.com/patent/US20220036178A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The disclosure herein describes training a global model based on a plurality of data sets. The global model is applied to each data set of the plurality of data sets and a plurality of gradients is generated based on that application. At least one gradient quality metric is determined for each gradient of the plurality of gradients. Based on the determined gradient quality metrics of the plurality of gradients, a plurality of weight factors is calculated. The plurality of gradients is transformed into a plurality of weighted gradients based on the calculated plurality of weight factors and a global gradient is generated based on the plurality of weighted gradients. The global model is updated based on the global gradient, wherein the updated global model, when applied to a data set, performs a task based on the data set and provides model output based on performing the task.

## Claims

### Claim 1

A system for training a global model based on a plurality of data sets, the system comprising:
at least one processor; and
at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the at least one processor to:
apply the global model to each data set of the plurality of data sets;
generate a plurality of gradients based on applying the global model to each data set of the plurality of data sets, wherein an individual gradient of the plurality of gradients is generated based on an individual data set of the plurality of data sets;
determine a plurality of gradient quality metrics, at least one gradient quality metric for each individual gradient of the plurality of gradients, the at least one gradient quality metric indicating a degree to which the individual gradient can be used to improve the global model;
calculate a plurality of weight factors using the determined plurality of gradient quality metrics, the plurality of weight factors including an individual weight factor for each individual gradient of the plurality of gradients;
transform the plurality of gradients into a plurality of weighted gradients based on the calculated plurality of weight factors;
generate a global gradient based on the plurality of weighted gradients; and
update the global model based on the global gradient, wherein the updated global model, when applied to the individual data set, performs a task based on the individual data set and provides model output based on performing the task.

### Claim 2 (depends on 1)

The system of claim 1, wherein the plurality of gradient quality metrics include at least one of gradient mean values, gradient variance values, or training loss values associated with applying the global model to each data set of the plurality of data sets.

### Claim 3 (depends on 1)

The system of claim 1, wherein calculating the plurality of weight factors includes applying a softmax function to the at least one gradient quality metric for each individual gradient of the plurality of gradients, wherein the individual weight factor of the plurality of weight factors for each individual gradient of the plurality of gradients is based on a result of applying the softmax function to the at least one gradient quality metric of the individual gradient.

### Claim 4 (depends on 3)

The system of claim 3, wherein calculating the plurality of weight factors further includes applying a weight factor neural network model to the at least one gradient quality metric for each individual gradient of the plurality of gradients, wherein the individual weight factor of the plurality of weight factors for each individual gradient of the plurality of gradients is based on a result of applying the weight factor neural network model to the at least one gradient quality metric of the individual gradient.

### Claim 5 (depends on 4)

The system of claim 4, wherein the plurality of weight factors includes a first set of weight factors based on applying the softmax function to the at least one gradient quality metric for each individual gradient of the plurality of gradients and a second set of weight factors based on applying the weight factor neural network model to the at least one gradient quality metric for each individual gradient of the plurality of gradients;
wherein the at least one memory and the computer program code configured to, with the at least one processor, further cause the at least one processor to:
test the first set of weight factors and the second set of weight factors based on a test data set; and
select a set of weight factors from the first set of weight factors and the second set of weight factors based on the testing; and
wherein transforming the plurality of gradients into a plurality of weighted gradients based on the calculated plurality of weight factors includes transforming the plurality of gradients into a plurality of weighted gradients based on the selected set of weight factors.

### Claim 6 (depends on 5)

The system of claim 5, wherein the at least one memory and the computer program code configured to, with the at least one processor, further cause the at least one processor to train the weight factor neural network model based on the testing of the first set of weight factors and the second set of weight factors based on the test data set.

### Claim 7 (depends on 1)

The system of claim 1, wherein the at least one memory and the computer program code configured to, with the at least one processor, further cause the at least one processor to, based on updating the global model based on the global gradient, train the updated global model using a held-out data set associated with the task, wherein divergence away from the task caused by updating the global model is reduced.

### Claim 8

A computerized method for training a global model based on a plurality of clients, the computerized method comprising:
providing, by a processor, the global model to a plurality of clients;
receiving, by the processor, a plurality of gradients from the plurality of clients;
determining, by the processor, a plurality of gradient quality metrics, at least one gradient quality metric for each individual gradient of the plurality of gradients, the at least one gradient quality metric indicating a degree to which the individual gradient can be used to improve the global model;
calculating, by the processor, a plurality of weight factors using the determined plurality of gradient quality metrics, the plurality of weight factors including an individual weight factor for each individual gradient of the plurality of gradients;
transforming, by the processor, the plurality of gradients into a plurality of weighted gradients based on the calculated plurality of weight factors;
generating, by the processor, a global gradient based on the plurality of weighted gradients; and
updating, by the processor, the global model based on the global gradient, wherein the updated global model, when applied to an individual data set, performs a task based on the individual data set and provides model output based on performing the task.

### Claim 9 (depends on 8)

The computerized method of claim 8, wherein the plurality of gradient quality metrics includes at least one of gradient mean values, gradient variance values, or training loss values associated with applying the global model to each data set of a plurality of data sets.

### Claim 10 (depends on 8)

The computerized method of claim 8, wherein calculating the plurality of weight factors includes applying, by the processor, a softmax function to the at least one gradient quality metric for each individual gradient of the plurality of gradients, wherein the individual weight factor of the plurality of weight factors for each individual gradient of the plurality of gradients is based on a result of applying the softmax function to the at least one gradient quality metric of the individual gradient.

### Claim 11 (depends on 10)

The computerized method of claim 10, wherein calculating the plurality of weight factors further includes applying, by the processor, a weight factor neural network model to the at least one gradient quality metric for each individual gradient of the plurality of gradients, wherein the individual weight factor of the plurality of weight factors for each individual gradient of the plurality of gradients is based on a result of applying the weight factor neural network model to the at least one gradient quality metric of the individual gradient.

### Claim 12 (depends on 11)

The computerized method of claim 11, wherein the plurality of weight factors includes a first set of weight factors based on applying the softmax function to the at least one gradient quality metric for each individual gradient of the plurality of gradients and a second set of weight factors based on applying the weight factor neural network model to the at least one gradient quality metric for each individual gradient of the plurality of gradients;
wherein the computerized method further includes:
testing, by the processor, the first set of weight factors and the second set of weight factors based on a test data set; and
selecting, by the processor, a set of weight factors from the first set of weight factors and the second set of weight factors based on the testing; and
wherein transforming the plurality of gradients into a plurality of weighted gradients based on the calculated plurality of weight factors includes transforming the plurality of gradients into a plurality of weighted gradients based on the selected set of weight factors.

### Claim 13 (depends on 12)

The computerized method of claim 12, further comprising training, by the processor, the weight factor neural network model based on the testing of the first set of weight factors and the second set of weight factors based on the test data set.

### Claim 14 (depends on 8)

The computerized method of claim 8, further comprising, based on updating the global model based on the global gradient, training, by the processor, the updated global model using a held-out data set associated with the task, wherein divergence away from the task caused by updating the global model is reduced.

### Claim 15

One or more non-transitory computer storage media having computer-executable instructions for applying a global model to a data set that, upon execution by a processor, cause the processor to at least:
access the data set;
apply the global model to the data set, whereby the global model performs a task on the accessed data set and provides model output based on performing the task;
train the global model based on the data set and the provided model output;
generate a gradient associated with the data set based on training the global model;
provide the generated gradient to a source of the global model, wherein the source of the global model is configured to:
include the generated gradient in a plurality of gradients associated with the global model;
calculate a plurality of weight factors, the plurality of weight factors including an individual weight factor for each individual gradient of the plurality of gradients; and
update the global model based on the plurality of gradients and plurality of weight factors; and
receive an updated global model from the source of the global model.

### Claim 16 (depends on 15)

The one or more non-transitory computer storage media of claim 15, wherein generating the gradient associated with the data set includes comparing a first version of the global model from prior to training the global model to a second version of the global model from after training the global model and generating the gradient based on differences in model parameters between the first version and second version.

### Claim 17 (depends on 15)

The one or more non-transitory computer storage media of claim 15, wherein training the global model based on the data set includes:
identifying data associated with the data set that indicates what accurate outputs of the global model should be; and
comparing the provided model output to the identified data.

### Claim 18 (depends on 17)

The one or more non-transitory computer storage media of claim 17, wherein the identified data includes at least one of manually defined classification labels of the data set or classification labels of the data generated based on another process.

### Claim 19 (depends on 15)

The one or more non-transitory computer storage media of claim 15, wherein the computer-executable instructions, upon execution by a processor, further cause the processor to at least apply the updated global model to the data set, whereby the updated global model performs a task on the accessed data set and provides model output based on performing the task.

### Claim 20 (depends on 15)

The one or more non-transitory computer storage media of claim 15, wherein the accessed data set is a batch portion of a larger set of data divided into a plurality of batch portions; and
wherein the computer-executable instructions, upon execution by a processor, further cause the processor to at least:
apply the global model to each batch portion of the plurality of batch portions whereby the global model performs a task on each batch portion and provides model output associated with each batch portion;
train the global model based on each batch portion and the associated model output;
generate a gradient associated with each batch portion based on training the global model; and
provide the generated gradient associated with each batch portion to a source of the global model.
