# US 2021/0241183 A1 · Adaptively synchronizing learning of multiple learning models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241183 A1
- Document ID: US20210241183A1
- Status: Published application
- Kind: A1
- Application number: 17/122,475
- Filed: 2020-12-15
- Published: 2021-08-05
- Earliest claimed: 2020-01-31
- Assignees: Hewlett Packard Enterprise Development LP
- Applicants: Hewlett Packard Enterprise Development LP
- Inventors: Vishesh GARG; Sathyanarayanan MANAMOHAN; Krishnaprasad Lingadahalli SHASTRY
- Web page: https://patentgrep.com/patent/US20210241183A1

## CPC

- G06N 20/00: Machine learning
- G06N 20/20: Ensemble learning
- G06F 9/3891: {organised in groups of units sharing resources, e.g. clusters}
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and a method for adaptively synchronizing learning of multiple learning models are disclosed. Several local learning models are executed on multiple nodes. Learning model parameters are shared by such nodes to a master node, in multiple iterations, after a predefined synchronization interval. Such learning model parameters are aggregated and central learning models are generated based on aggregated set of learning model parameters. Accuracies of the central learning models and an average accuracy of the central learning models are determined. Accuracy of an immediate central learning model i.e. the one received after determining the average accuracy, is compared with the average accuracy. Based on the difference between the accuracy of the immediate central learning model and the average accuracy, the synchronization interval is modified, and the multiple nodes are updated about this modified synchronization interval.

## Claims

### Claim 1

A system, comprising:
a plurality of data processing nodes programmed to run a plurality of local learning models; and
a master node programmed to communicate with the plurality of data processing nodes to:
aggregate a set of learning model parameters received iteratively from the data processing nodes, after a predefined synchronization interval, wherein the learning model parameters are received for a predefined number of iterations;
generate central learning models based on the aggregated set of learning model parameters received after each of the predefined number of iterations;
determine accuracies of the central learning models;
determine an average accuracy by averaging the accuracies of the central learning models;
determine a difference between an accuracy of an immediate central learning model and the average accuracy;
update an accuracy metric based on the difference between the accuracy of the immediate central model and the average accuracy;
compare the accuracy metric to a range established by a set of thresholds; and
when the accuracy metric is outside of the range established by the set of thresholds, modify the predefined synchronization interval.

### Claim 2 (depends on 1)

The system as claimed in claim 1, wherein the accuracy metric is represented as Mean Squared Errors (MSEs).

### Claim 3 (depends on 1)

The system as claimed in claim 1, wherein the accuracies of the central learning models and the accuracy of the immediate central learning model are determined based on results obtained by implementation of the central learning models and the immediate central learning model on pre-stored evaluation data.

### Claim 4 (depends on 1)

The system as claimed in claim 1, wherein the synchronization interval is down-scaled by a predefined scaling factor when the accuracy metric falls below the range established by the set of thresholds.

### Claim 5 (depends on 1)

The system as claimed in claim 1, wherein the synchronization interval is up-scaled by a predefined scaling factor when the accuracy metric exceeds the range established by the set of thresholds.

### Claim 6 (depends on 1)

The system as claimed in claim 1, wherein the master node and the plurality of data processing nodes are a part of a decentralized data processing network or a distributed data processing network.

### Claim 7

A method, comprising:
periodically receiving, at a master node of a plurality of data processing nodes, a set of learning model parameters from each node of the plurality of data processing nodes,
wherein the each successive set of learning model parameters from the respective nodes are iteratively received after a predefined synchronization interval, and wherein the learning model parameters are received for a predefined number of iterations;
aggregating, after each of the predefined number of iterations, the set of learning model parameters received after the predefined synchronization interval;
generating central learning models based on the aggregated set of learning model parameters received after each of the predefined number of iterations;
determining accuracies of the central learning models;
determining an average accuracy by averaging the accuracies of the central learning models;
determining a difference between an accuracy of an immediate central learning model and the average accuracy;
comparing the difference in accuracy to a set of thresholds;
when the difference in accuracy is outside of a range established by the set of thresholds, modifying the predefined synchronization interval; and
transmitting, to the plurality of data processing nodes, the modified synchronization interval.

### Claim 8 (depends on 7)

The method as claimed in claim 7, wherein the accuracy is represented as Mean Squared Errors (MSEs).

### Claim 9 (depends on 7)

The method as claimed in claim 7, wherein the accuracies of the central learning models and the accuracy of the immediate central learning model are determined based on results obtained by implementation of the central learning models and the immediate central learning model on pre-stored evaluation data.

### Claim 10 (depends on 7)

The method as claimed in claim 7, wherein the synchronization interval is modified by the master node elected amongst the plurality of data processing nodes.

### Claim 11 (depends on 7)

The method as claimed in claim 7, wherein the synchronization interval is down-scaled by a predefined scaling factor when the average difference in accuracy falls below the range established by the set of thresholds.

### Claim 12 (depends on 7)

The method as claimed in claim 7, wherein the synchronization interval is up-scaled by a predefined scaling factor when the average difference in accuracy exceeds the range established by the set of thresholds.

### Claim 13 (depends on 7)

The method as claimed in claim 7, wherein the master node and the plurality of data processing nodes are a part of a decentralized data processing network or a distributed data processing network.

### Claim 14

A non-transitory machine-readable storage medium that stores instructions which, when executed by a processor, perform a plurality of operations comprising:
receiving a set of learning model parameters iteratively from each node of a plurality of data processing nodes,
wherein the set of learning model parameters is received after a predefined synchronization interval, and wherein the learning model parameters are received for a predefined number of iterations;
generating central learning models based on the set of learning model parameters received after each of the predefined number of iterations;
determining accuracies of the central learning models;
determining an average accuracy by averaging the accuracies of the central learning models;
determining a difference between an accuracy of an immediate central learning model and the average accuracy;
comparing the difference in accuracy to a range established by a set of thresholds;
when the difference in accuracy is outside of the range, modifying the predefined synchronization interval; and
transmitting, to the plurality of data processing nodes, the modified synchronization interval.
