Systems and methods for meta-learning in federated learning networks

A method of federated learning performed by a server and a plurality of client devices, the method comprising:

• receiving, at each client device, a global model comprising a set of model parameters maintained by the server;
• performing, at each client device, a plurality of local fine-tuning steps on the global model using local training data of the client device to generate a personalized model for the client device;
• computing, at each client device, update information based on the personalized model using a first-order approximation that avoids computation of second-order derivatives;
• transmitting, from each client device to the server, the update information;
• and
• updating, at the server, the global model based on the update information received from the client devices,
• wherein the update information is computed using a Hessian-free approximation of a meta-gradient associated with the personalized model after the plurality of local fine-tuning steps.

A system comprising:

• a server configured to maintain a global model comprising a set of model parameters;
• and
• a plurality of client devices, each client device configured to:
• receive the global model from the server;
• perform a plurality of local fine-tuning steps on the global model using local training data of the client device to generate a personalized model;
• and
• compute update information based on the personalized model after the plurality of local fine-tuning steps,
• wherein the plurality of client devices are further configured to transmit the update information to the server, and
• wherein the server is further configured to update the global model based on the update information received from the plurality of client devices, the updating being based at least in part on a loss of the personalized models evaluated after the plurality of local fine-tuning steps.