Sketched and clustered federated learning with automatic tuning

A computer program product for automatic adaptive client selection in federated learning, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors, the program instructions executable to:

• send, by a server in the federated learning, to all of clients in the federated learning, parameters of a machine learning model, wherein all of the clients compute respective gradients using the parameters;
• receive, by the server from all of the clients, sketches of the respective gradients, wherein the sketches are computed by all of the clients;
• use, by the server, the sketches to compute similarity between all of the clients;
• cluster, by the server, all of the clients, based on the similarity;
• optimize, by the server, a number of client clusters and a dimension of the sketches, subject to a constraint of memory consumption, a constraint of communication overhead, and a performance metric of the federated learning;
• determine, by the server, a subset of the clients that send the respective gradients to the server, by selecting the clients from respective ones of the client clusters;
• and
• aggregate, by the server, the respective gradients sent by the subset of the clients.

A computer system for automatic adaptive client selection in federated learning, the computer system comprising one or more processors, one or more computer readable tangible storage devices, and program instructions stored on at least one of the one or more computer readable tangible storage devices for execution by at least one of the one or more processors, the program instructions executable to:

• send, by a server in the federated learning, to all of clients in the federated learning, parameters of a machine learning model, wherein all of the clients compute respective gradients using the parameters;
• receive, by the server from all of the clients, sketches of the respective gradients, wherein the sketches are computed by all of the clients;
• use, by the server, the sketches to compute similarity between all of the clients;
• cluster, by the server, all of the clients, based on the similarity;
• optimize, by the server, a number of client clusters and a dimension of the sketches, subject to a constraint of memory consumption, a constraint of communication overhead, and a performance metric of the federated learning;
• determine, by the server, a subset of the clients that send the respective gradients to the server, by selecting the clients from respective ones of the client clusters;
• and
• aggregate, by the server, the respective gradients sent by the subset of the clients.