US 12,748,982 B2Granted

Personalized federated learning via sharable basis models

AssigneeGoogle Inc.

InventorsHong-You Chen, Boqing Gong, Mingda Zhang, Hang Qi, Xuhui Jia, Li Zhang

Life of this patent

Estimating the term…

Loading patent…

Abstract

The embodiments are directed towards providing personalized federated learning (PFL) models via sharable federated basis models. A model architecture and learning algorithm for PFL models is disclosed. The embodiments learn a set of basis models, which can be combined layer by layer to form a personalized model for each client using specifically learned combination coefficients. The set of basis models are shared with each client of a set of the clients. Thus, the set of basis models is common to each client of the set of clients. However, each client may generate a unique PFL based on their specifically learned combination coefficients. The unique combination of coefficients for each client may be encoded in a separate personalized vector for each of the clients.

Claims20

Dependent claims

A method implemented by a server device, the method comprising:

providing, each client device of a set of client devices, a set of untrained models;
causing, each client device of the set of client devices, to generate a separate set of trained models based on the set of untrained models, wherein each client device iteratively trains the set of untrained models based on a separate subset of a set of training data that is located locally on the client device such that each subset of the set of training data is inaccessible by the server device and each subset of the set of training data is inaccessible by the client device except for the subset of training data that is located locally on the client device;
receiving, at the server device, a separate set of trained models from each client device;
generating, at the server device, a set of basis models based on a combination of the separate set of trained models received from each of the client devices;
providing, each client of the set of clients, the set of basis models;
and
causing, each client device of the set of client devices, to generate a personalized model based on a separate linear combination of the basis models of the set of basis models.

4 more elements

A computing system comprising:

one or more processors;
and
one or more non-transitory computer-readable media that store instructions that when executed by the one or more processors, cause the computer system to perform operations comprising:
providing, each client device of a set of client devices, a set of untrained models;
causing, each client device of the set of client devices, to generate a separate set of trained models based on the set of untrained models, wherein each client device iteratively trains the set of untrained models based on a separate subset of a set of training data that is located locally on the client device such that each subset of the set of training data is inaccessible by the computing system and each subset of the set of training data is inaccessible by the client device except for the subset of training data that is located locally on the client device;
receiving, at the computing system, a separate set of trained models from each client device;
generating, at the computing system, a set of basis models based on a combination of the separate set of trained models received from each of the client devices;
providing, each client of the set of clients, the set of basis models;
and causing, each client device of the set of client devices, to generate a personalized model based on a separate linear combination of the basis models of the set of basis models.

6 more elements

Dependencies are inferred from the claim text.