# US 2026/0087416 A1 · Apparatus and methods for federated learning, device and method for a device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087416 A1
- Document ID: US20260087416A1
- Status: Published application
- Kind: A1
- Application number: 19/096,113
- Filed: 2025-03-31
- Published: 2026-03-26
- Earliest claimed: 2024-09-23
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Weiming ZHUANG; Jingtao LI; Lingjuan LYU
- Web page: https://patentgrep.com/patent/US20260087416A1

## CPC

- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 20/20: Ensemble learning
- G06N 3/096: Transfer learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/094: Adversarial learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The federated learning of a first machine-learning model apparatus includes processing circuitry configured to generate a second machine-learning model including a backbone and a decoder from the first machine-learning model. The processing circuitry is configured to perform at least one iteration of the following: (a) output the decoder of the second machine-learning model to one or more devices and for the first iteration of the at least one iteration further output the backbone of the second machine-learning model to one or more devices; (b) receive a trained version of a decoder for the second machine-learning model from one or more devices; and (c) update the decoder of the second machine-learning model based on the trained version of the decoder received from one or more devices. The processing circuitry is configured to update a decoder of the first machine-learning model based on the updated decoder of the second machine-learning model.

## Claims

### Claim 1

An apparatus for federated learning of a first machine-learning model, the apparatus comprising processing circuitry configured to:
generate a second machine-learning model from the first machine-learning model, wherein the second machine-learning model comprises a backbone and a decoder;
perform at least one iteration of the following (a) to (c):
(a) output the decoder of the second machine-learning model to one or more devices and for the first iteration of the at least one iteration further output the backbone of the second machine-learning model to the one or more devices;
(b) receive a trained version of a decoder for the second machine-learning model from the one or more devices; and
(c) update the decoder of the second machine-learning model based on the trained version of the decoder received from the one or more devices; and
update a decoder of the first machine-learning model based on the updated decoder of the second machine-learning model.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is further configured to iteratively perform (a) to (c) until the second machine-learning model with the updated decoder satisfies a predefined criterion.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to update only the decoder of the first machine-learning model while keeping a backbone of the first machine-learning model unchanged.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to control the one or more devices to train only the decoder of the second machine-learning model locally at the one or more devices using local data at the respective device while keeping the backbone of the second machine-learning model unchanged.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to update the decoder of the first machine-learning model by replacing the decoder of the first machine-learning model with the updated decoder of the second machine-learning model.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the second machine-learning model is smaller than the first machine-learning model.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the processing circuitry is configured to generate the second machine-learning model from the first machine-learning model using knowledge distillation.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the processing circuitry is configured to generate the second machine-learning model from the first machine-learning model using knowledge distillation by training the backbone of the second machine-learning model to minimize a loss function that measures the difference between output data of the backbone of the second machine-learning model and output data of a backbone of the first machine-learning model for the same input data.

### Claim 9 (depends on 6)

The apparatus of claim 6, wherein the second machine-learning model is smaller with respect to at least one of complexity, size and resource requirements compared to the first machine-learning model.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to keep the first machine-learning model unchanged when generating the second machine-learning model.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to update the decoder of the first machine-learning model based on the updated decoder of the second machine-learning model obtained in the last iteration of the at least one iteration.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the first machine-learning model is a foundation model.

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein, for generating the second machine-learning model from the first machine-learning model, the processing circuitry is configured to perform supervised training of a decoder of the first machine-learning model while keeping a backbone of the first machine-learning model unchanged, wherein the supervised training is performed using local data at the apparatus.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the processing circuitry is configured to control the one or more devices to train only the decoder of the second machine-learning model locally at the one or more devices while keeping a backbone of the second machine-learning model unchanged, wherein the training is performed using local data at the respective device having a lower resolution than the local data at the apparatus used for the supervised training.

### Claim 15 (depends on 1)

The apparatus of claim 1, wherein, for updating the decoder of the second machine-learning model based on the trained version of the decoder received from the one or more devices, the processing circuitry is configured to:
aggregate the trained versions of the decoder received from multiple devices to generate an aggregated decoder of the second machine-learning model; and
perform supervised training of the aggregated decoder of the second machine-learning model using local data at the apparatus.

### Claim 16 (depends on 1)

The apparatus of claim 1, wherein, for updating the decoder of the second machine-learning model based on the trained version of the decoder received from the one or more devices, the processing circuitry is configured to:
aggregate the trained versions of the decoder received from multiple devices to generate an aggregated decoder of the second machine-learning model;
combine the aggregated decoder of the second machine-learning model with the decoder of the second machine-learning model obtained in the previous iteration to generate a combined decoder of the second machine-learning model; and
perform supervised training of the combined decoder of the second machine-learning model using local data at the apparatus.

### Claim 17 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is further configured to iteratively perform (a) to (c) for a predefined number of iterations.

### Claim 18 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to randomly select the one or more devices for each iteration from a plurality of available devices.

### Claim 19 (depends on 1)

A server or a computing cloud comprising the apparatus according to claim 1.

### Claim 20

A device comprising processing circuitry configured to perform at least one iteration of the following:
receive a decoder of a machine-learning model from a server or computing cloud, wherein a backbone of the machine-learning model is further received in the first iteration of the at least one iteration, and wherein the received decoder is an updated version of the received decoder compared to a previous iteration for the second and each further iteration of the at least one iteration;
train the received decoder of the machine-learning model using local data at the device; and
output the trained decoder for the machine-learning model to the server or computing cloud.

### Claim 21 (depends on 20)

The device of claim 20, wherein the processing circuitry is configured to train only the received decoder of the machine-learning model using the local data at the device while keeping the backbone of the machine-learning model unchanged.

### Claim 22 (depends on 20)

The device of claim 20, wherein the processing circuitry is configured to output only the trained decoder for the machine-learning model to the server or computing cloud.

### Claim 23 (depends on 20)

The device of claim 20, wherein the processing circuitry is configured to train the received decoder of the machine-learning model unsupervised.

### Claim 24 (depends on 20)

The device of claim 20, wherein the local data at the device are unlabeled, and wherein, for training the received decoder of the machine-learning model, the processing circuitry is configured to:
generate a teacher model and a student model based on the received decoder of the machine-learning model;
generate pseudo labels for the local data at the device using the teacher model;
train a decoder of the student model based on the generated pseudo labels; and
update a decoder of the teacher model using an exponential moving average of weights of the decoder of the student model,
wherein the trained decoder for the machine-learning model output to the server or computing cloud is the trained decoder of the student model.

### Claim 25

A method for a device, wherein the method comprises performing at least one iteration of the following:
receiving a decoder of a machine-learning model from a server or computing cloud, wherein a backbone of the machine-learning model is further received in the first iteration of the at least one iteration, and wherein the received decoder is updated version of the received decoder compared to a previous iteration for the second and each further iteration of the at least one iteration;
training the received decoder of the machine-learning model using local data at the device; and
outputting the trained decoder for the machine-learning model to the server or computing cloud.
