Computation-efficient federated learning for systems with resource heterogeneity

A computer-implemented method for training a global model on a central server in a federated learning system having a plurality of nodes, the method comprising:

• obtaining a number of complexity levels for the federated learning system;
• obtaining a target computational overhead reduction ratio for each of the complexity levels;
• splitting the global model along a width and a depth via two-dimensional uniform downscaling of the global model to create a plurality of local models, wherein one of the plurality of local models corresponds to each of the number of complexity levels;
• computing a computational overhead of each of a plurality of local models at each of the complexity levels;
• sending an assigned one of the plurality of local models to each of a plurality of clients based on an available computational overhead budget at each of the plurality of clients, wherein the computational overhead of the assigned one is less than the available computational overhead budget at each of the plurality of clients;
• and
• training the assigned ones of the plurality of local models on respective ones of the plurality of clients.

A non-transitory computer readable storage medium tangibly embodying a computer readable program code having computer readable instructions that, when executed, causes a computer device to carry out a method of training a global model on a central server in a federated learning system having a plurality of nodes, the method comprising:

• splitting the global model along a width and a depth via two-dimensional uniform downscaling of the global model;
• creating a plurality of local models based on the splitting of the global model;
• and
• training selected ones of the plurality of local models on respective selected ones of a plurality of clients based on computational constraints of each of the plurality of clients.