# US 2025/0028996 A1 · An adaptive personalized federated learning method supporting heterogeneous model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0028996 A1
- Document ID: US20250028996A1
- Status: Published application
- Kind: A1
- Application number: 18/281,938
- Filed: 2023-03-17
- Published: 2025-01-23
- Earliest claimed: 2022-08-01
- Assignees: Zhejiang University; Zhejiang University Zhongyuan Institute
- Applicants: Zhejiang University; Zhejiang University Zhongyuan Institute
- Inventors: SHUIGUANG DENG; ZHEN QIN
- Web page: https://patentgrep.com/patent/US20250028996A1

## CPC

- G06N 20/00: Machine learning
- G06N 3/08: Learning methods
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present invention discloses an adaptive personalized federated learning method supporting heterogeneous model, based on the use of models with different structures by various participants supporting federated learning, learning the dynamic weight used for model ensemble and introducing optimization objectives for model integration in the process of training model parameters, realizing highly accurate personalized federated learning with heterogeneous and self adaptive data, the participants are enabled to benefit from federated learning in scenes with heterogeneous data at different levels. The adaptive personalized federated learning method of the present invention does not need to introduce new hyper parameters, and can be conveniently deployed in the existing federated learning system; comparing with the traditional personalized federated learning method, the present invention has stronger adaptability.

## Claims

### Claim 1

An adaptive personalized federated learning method supporting heterogeneous model, comprising the following steps:
(1) initializing parameters of a global shared model by a central server;
(2) the central server sending the parameters of the global shared model to each participant of the federated learning, after receiving the parameters of the global shared model, the participants updating their own global shared model with the parameters;
(3) the participants performing learning for adaptability to update the weights of private models;
(4) the participants using newly obtained private training data to train both the private models and the globally shared model by using a stochastic gradient descent algorithm;
(5) the participants uploading the parameters of the globally shared model to the central server after one round of iterative training;
(6) after collecting enough parameters of the global shared model, the central server aggregating these model parameters to obtain new parameters of the global shared model, and then returning to step (2) to distribute the new parameters of the global shared model to each participant, and then circulating until the loss function of all models converges or reaches the maximum number of iterations in federated learning;
wherein, the private models are the models held by each participant of federated learning, and the structure and parameters are not disclosed, the structure of the private models held by each participant are different;
wherein, the specific implementation of step (3) is as follows: the participants first dividing a small portion (such as 5% of the training data) from the obtained private training data as a validation set, and inferring the private models and the global shared model on the validation set, obtaining the prediction output result ppri of the private models and the prediction output result psha of the global shared models, then the participants updating the weight of the private models through the stochastic gradient descent method, and the update expression is as follows:
λ
i
′
=
λ
i
-
η
⁢
∇
λ
i
L
C
⁢
E
(
p
aen
,
y
)
wherein, λi is the weight of the private model before update, λ′i is the weight of the private model after update, n represents learning rate, ∇λi represents LCE (paen y) to λi gradient, LCE (paen, Y) represents cross entropy of paen and y, paen represents the weighted average result of ppri and psha, y is the ground-truth label;
wherein, the loss function expression used for private model training in step (4) is as follows:
L
p
⁢
r
⁢
i
=
L
C
⁢
E
(
P
pri
,
y
)
+
D
K
⁢
L
(
p
p
⁢
r
⁢
i
⁢

p
s
⁢
h
⁢
a
)
+
L
C
⁢
E
(
p
aen
,
y
)
wherein, Lpri is the loss function of the private model, LCE(ppri, y) represents cross entropy of ppri and y, LCE (paen, y) represents cross entropy of paen and y, DKL (ppri∥psha) represents the KL divergence of ppri relative to psha, paen represents the weighted average result of ppri and psha, y is the ground-truth label, psha is the prediction output result of the global shared model;
wherein, the expression of the loss function used for the global shared model training in step (4) is as follows:
L
s
⁢
h
⁢
a
=
L
C
⁢
E
(
p
s
⁢
h
⁢
a
,
y
)
+
D
K
⁢
L
(
p
s
⁢
h
⁢
a
||
p
p
⁢
r
⁢
i
)
+
L
C
⁢
E
(
p
aen
,
y
)
wherein, Lsha is the loss function of the global shared model, LCE(psha,y) represents cross entropy of psha and y, LCE(paen, y) represents cross entropy of paen and y, DKL (psha∥ppri) represents the KL divergence of psha relative to ppri, paen represents the weighted average result of ppri and psha, y is the ground-truth label, ppri is the prediction output result of the private model, psha is the prediction output result of the global shared model.

### Claim 2 (depends on 1)

The adaptive personalized federated learning method according to claim 1, wherein, the global shared model is trained by the participants of federated learning, and the central server is responsible for aggregation, each participant holds a copy of the global shared model; on the one hand, the global shared model is used for inference by each participant after the completion of federated learning training, and on the other hand, the global shared model serves as a medium for knowledge sharing among participants.

### Claim 3

(canceled)

### Claim 4 (depends on 1)

The adaptive personalized federated learning method according to claim 1, wherein, the participants are end devices in the federated learning system, in order to profit from the federated learning system, that is, to obtain a model with more accuracy, the participants upload model parameters to the central server and downloads aggregated model parameters from the central server.

### Claim at position 5

5-7. (canceled)

### Claim 8 (depends on 1)

The adaptive personalized federated learning method according to claim 1, wherein, in step (6), after collecting sufficient global shared models from the clients, the central server executes a federated averaging algorithm to aggregate the received models, and then distributes the aggregated new global shared model to all participants.