Concepts for distributed learning of neural networks and/or transmission of parameterization updates therefor

A system for federated learning of a neural network in cycles, the system comprising a server and clients and configured to, in each cycle

• download, from the server to a predetermined client, information on a setting of a parameterization of the neural network,
• the predetermined client,
• updating the setting of the parameterization of the neural network using training data at least partially individually gathered by the respective client to acquire a parameterization update, and
• uploading information on the parameterization update,
• merge, by the server, the parameterization update with further parametrization updates of other clients to acquire a merged parameterization update defining a further setting for the parameterization for a subsequent cycle,
• wherein the uploading of the information on the parameterization update comprises lossy coding of an accumulated parametrization update corresponding to a first accumulation of the parameterization update of a current cycle on the one hand and coding losses of uploads of information on parameterization updates of previous cycles on the other hand.

A client device for decentralized training contribution to federated learning of a neural network in cycles, the client device being configured to, in each cycle,

• receive information on a setting of a parameterization of the neural network,
• gather training data,
• update the setting of the parameterization of the neural network using the training data to acquire a parameterization update, and
• uploading information on the parameterization update for being merged with the parameterization updates of other clients devices to acquire a merged parameterization update defining a further setting of the parameterization for a subsequent cycle,
• wherein the client device is configured to, in uploading the information on the parameterization update, lossy code an accumulated parametrization update corresponding to a first accumulation of the parameterization update of a current cycle on the one hand and coding losses of uploads of information on parameterization updates of previous cycles on the other hand.

A method for decentralized training contribution to federated learning of a neural network in cycles, the method comprising, in each cycle,

• receiving information on a setting of a parameterization of the neural network,
• gathering training data,
• updating the setting of the parameterization of the neural network using the training data to acquire a parameterization update, and
• uploading information on the parameterization update for being merged with the parameterization updates of other clients deices to acquire a merged parameterization update defining a further setting of the parameterization for a subsequent cycle,
• wherein the method comprises, in uploading the information on the parameterization update, lossy coding an accumulated parametrization update corresponding to a first accumulation of the parameterization update of a current cycle on the one hand and coding losses of uploads of information on parameterization updates of previous cycles on the other hand.

A method for distributed learning of a neural network by clients in cycles, the method comprising, in each cycle

• downloading, to a predetermined client, information on a setting of a parameterization of the neural network,
• the predetermined client
• updating the setting of the parameterization of the neural network using training data to acquire a parameterization update, and
• uploading information on the parameterization update,
• merging the parameterization update with further parametrization updates of the other clients to acquire a merged parameterization update which defines a further setting of the parameterization for a subsequent cycle,
• wherein, in a predetermined cycle, the downloading the information on the setting of the parameterization of the neural network comprises downloading information on the merged parametrization update of a preceding cycle by lossy coding of an accumulated merged parametrization update corresponding to a first accumulation of the merged parametrization update of the preceding cycle on the one hand and coding losses of downloads of information on merged parametrization updates of cycles preceding the preceding cycle on the other hand.

A system for distributed learning of a neural network in cycles, the system comprising a server and clients and configured to, in each cycle

• download, from the server to a predetermined client, information on a setting of a parameterization of the neural network,
• the predetermined client
• updating the setting of the parameterization of the neural network using training data to acquire a parameterization update, and
• uploading information on the parameterization update,
• merge, by the server, the parameterization update with further parametrization updates of the other clients to acquire a merged parameterization update which defines a further setting of the parameterization for a subsequent cycle,
• wherein, in a predetermined cycle, the downloading the information on the setting of the parameterization of the neural network comprises downloading information on the merged parametrization update of a preceding cycle by lossy coding of an accumulated merged parametrization update corresponding to a first accumulation of the merged parametrization update of the preceding cycle on the one hand and coding losses of downloads of information on merged parametrization updates of cycles preceding the preceding cycle on the other hand.

An apparatus for coordinating a distributed learning of a neural network by clients in cycles, the apparatus configured to, per cycle,

• download, to a predetermined client, information on a setting of a parameterization of the neural network for sake of the clients updating the setting of the parameterization of the neural network using training data to acquire a parameterization update,
• receive information on the parameterization update from the predetermined client,
• merge the parameterization update with further parametrization updates from other clients to acquire a merged parameterization update which defines a further setting of the parameterization for a subsequent cycle,
• wherein the apparatus is configured to, in a predetermined cycle, in downloading the information on the setting of the parameterization of the neural network, download the merged parametrization update of a preceding cycle by lossy coding of an accumulated merged parametrization update corresponding to a first accumulation of the merged parametrization update of the preceding cycle on the one hand and coding losses of downloads of information on merged parametrization updates of cycles preceding the preceding cycle on the other hand.

A method for coordinating a distributed learning of a neural network by clients in cycles, the method comprising, per cycle,

• downloading, to a predetermined client, information on a setting of a parameterization of the neural network for sake of the clients updating the setting of the parameterization of the neural network using training data to acquire a parameterization update,
• receiving information on the parameterization update from the predetermined client,
• merging the parameterization update with further parametrization updates from other clients to acquire a merged parameterization update which defines a further setting of the parameterization for a subsequent cycle,
• wherein the method comprises, in a predetermined cycle, in downloading the information on the setting of the parameterization of the neural network, downloading the merged parametrization update of a preceding cycle by lossy coding of an accumulated merged parametrization update corresponding to a first accumulation of the merged parametrization update of the preceding cycle on the one hand and coding losses of downloads of information on merged parametrization updates of cycles preceding the preceding cycle on the other hand.

A non-transitory digital storage medium having stored thereon a computer program for performing a method for federated learning of a neural network by clients in cycles, the method comprising in each cycle

• downloading, to a predetermined client, information on a setting of a parameterization of the neural network,
• the predetermined client,
• updating the setting of the parameterization of the neural network using training data at least partially individually gathered by the respective client to acquire a parameterization update, and
• uploading information on the parameterization update,
• merging the parameterization update with further parametrization updates of other clients to acquire a merged parameterization update defining a further setting for the parameterization for a subsequent cycle,
• wherein the uploading of the information on the parameterization update comprises lossy coding of an accumulated parametrization update corresponding to a first accumulation of the parameterization update of a current cycle on the one hand and coding losses of uploads of information on parameterization updates of previous cycles on the other hand,
• when said computer program is run by a computer.

A non-transitory digital storage medium having stored thereon a computer program for performing a method for coordinating a distributed learning of a neural network by clients in cycles, the method comprising, per cycle,

• downloading, to a predetermined client, information on a setting of a parameterization of the neural network for sake of the clients updating the setting of the parameterization of the neural network using training data to acquire a parameterization update,
• receiving information on the parameterization update from the predetermined client,
• merging the parameterization update with further parametrization updates from other clients to acquire a merged parameterization update which defines a further setting of the parameterization for a subsequent cycle,
• wherein the method comprises, in a predetermined cycle, in downloading the information on the setting of the parameterization of the neural network, downloading the merged parametrization update of a preceding cycle by lossy coding of an accumulated merged parametrization update corresponding to a first accumulation of the merged parametrization update of the preceding cycle on the one hand and coding losses of downloads of information on merged parametrization updates of cycles preceding the preceding cycle on the other hand,
• when said computer program is run by a computer.