# US 2026/0087371 A1 · Apparatus and methods for federated learning of a first machine-learning model, device and method for a device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087371 A1
- Document ID: US20260087371A1
- Status: Published application
- Kind: A1
- Application number: 18/893,069
- Filed: 2024-09-23
- Published: 2026-03-26
- Earliest claimed: 2024-09-23
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Weiming ZHUANG; Jingtao LI; Lingjuan LYU
- Web page: https://patentgrep.com/patent/US20260087371A1

## CPC

- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 3/045: Combinations of networks
- G06N 3/096: Transfer learning
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/094: Adversarial learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The federated learning of a first machine-learning model apparatus includes processing circuitry configured to generate a smaller second machine-learning model including a backbone and a decoder from the first machine-learning model. The processing circuitry is configured to perform at least one iteration of the following: (a) output the decoder of the second machine-learning model to one or more devices and for the first iteration of the at least one iteration further output the backbone of the second machine-learning model to one or more devices; (b) receive a trained version of a decoder for the second machine-learning model from one or more devices; and (c) update the decoder of the second machine-learning model based on the trained version of the decoder received from one or more device. The processing circuitry is configured to update a decoder of the first machine-learning model based on the updated decoder of the second machine-learning model.

## Claims

### Claim 1

An apparatus for federated learning of a first machine-learning model, the apparatus comprising processing circuitry configured to:
generate a second machine-learning model from the first machine-learning model, wherein the second machine-learning model is smaller than the first machine-learning model, and wherein the second machine-learning model comprises a backbone and a decoder;
perform at least one iteration of the following (a) to (c):
(a) output the decoder of the second machine-learning model to one or more devices and for the first iteration of the at least one iteration further output the backbone of the second machine-learning model to the one or more devices;
(b) receive a trained version of a decoder for the second machine-learning model from the one or more devices; and
(c) update the decoder of the second machine-learning model based on the trained version of the decoder received from the one or more devices; and
update a decoder of the first machine-learning model based on the updated decoder of the second machine-learning model.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is further configured to iteratively perform (a) to (c) until the second machine-learning model with the updated decoder satisfies a predefined criterion.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to update only the decoder of the first machine-learning model while keeping a backbone of the first machine-learning model unchanged.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to control the one or more devices to train only the decoder of the second machine-learning model locally at the one or more devices using local data at the respective device while keeping the backbone of the second machine-learning model unchanged.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to update the decoder of the first machine-learning model by replacing the decoder of the first machine-learning model with the updated decoder of the second machine-learning model.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to generate the second machine-learning model from the first machine-learning model using knowledge distillation.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the processing circuitry is configured to generate the second machine-learning model from the first machine-learning model using knowledge distillation by training the backbone of the second machine-learning model to minimize a loss function that measures the difference between output data of the backbone of the second machine-learning model and output data of a backbone of the first machine-learning model for the same input data.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to keep the first machine-learning model unchanged when generating the second machine-learning model.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is configured to update the decoder of the first machine-learning model based on the updated decoder of the second machine-learning model obtained in the last iteration of the at least one iteration.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the second machine-learning model is smaller with respect to at least one of complexity, size and resource requirements compared to the first machine-learning model.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the first machine-learning model is a foundation model.

### Claim 12 (depends on 1)

A server or a computing cloud comprising the apparatus according to claim 1.

### Claim 13

A device comprising processing circuitry configured to perform at least one iteration of the following:
receive a decoder of a machine-learning model from a server or computing cloud, wherein a backbone of the machine-learning model is further received in the first iteration of the at least one iteration, and wherein the received decoder is an updated version of the received decoder compared to a previous iteration for the second and each further iteration of the at least one iteration;
train the received decoder of the machine-learning model using local data at the device; and
output the trained decoder for the machine-learning model to the server or computing cloud.

### Claim 14 (depends on 13)

The device of claim 13, wherein the processing circuitry is configured to train only the received decoder of the machine-learning model using the local data at the device while keeping the backbone of the machine-learning model unchanged.

### Claim 15 (depends on 13)

The device of claim 13, wherein the processing circuitry is configured to output only the trained decoder for the machine-learning model to the server or computing cloud.

### Claim 16

A method for federated learning of a first machine-learning model, the method comprising:
generating a second machine-learning model from the first machine-learning model, wherein the second machine-learning model is smaller than the first machine-learning model, wherein the second machine-learning model comprises a backbone and a decoder;
performing at least one iteration of the following (a) to (c):
(a) outputting the decoder of the second machine-learning model to one or more devices and for the first iteration of the at least one iteration further outputting the backbone of the second machine-learning model to the one or more devices;
(b) receiving a trained version of a decoder for the second machine-learning model from the one or more devices; and
(c) updating the decoder of the second machine-learning model based on the trained version of the decoder received from the one or more devices; and
updating a decoder of the first machine-learning model based on the updated decoder of the second machine-learning model.

### Claim 17 (depends on 16)

The method of claim 16, wherein the method is performed by a server or a computing cloud.

### Claim 18 (depends on 16)

The method of claim 16, wherein (a) to (c) are iteratively performed until the second machine-learning model with the updated decoder satisfies a predefined criterion.

### Claim 19

A method for a device, wherein the method comprises performing at least one iteration of the following:
receiving a decoder of a machine-learning model from a server or computing cloud, wherein a backbone of the machine-learning model is further received in the first iteration of the at least one iteration, and wherein the received decoder is updated version of the received decoder compared to a previous iteration for the second and each further iteration of the at least one iteration;
training the received decoder of the machine-learning model using local data at the device; and
outputting the trained decoder for the machine-learning model to the server or computing cloud.

### Claim 20 (depends on 19)

The method of claim 19, wherein only the received decoder of the machine-learning model is trained using the local data at the device while the backbone of the machine-learning model is kept unchanged.
