Apparatuses, methods and computer program products for routing data using multi-agent federated learning

A user device comprising:

• at least one processor;
• and
• at least one memory storing instructions that, when executed by the at least one processor, cause the user device at least to:
• receive a packet data unit, PDU, of one or more incoming PDUs to be transmitted by the user device, via a respective path, to one amongst a master node and at least one secondary node;
• receive, once entered a network, an aggregated Q-table from a central entity of the network, the aggregated Q-table including rewards and being shared with the user device and with other user devices also located in the network;
• perform a local exploration during a certain time interval using the aggregated Q-table, and generate, in response to the local exploration, a local Q-table, the local Q-table including rewards;
• compare the respective rewards from the aggregated Q-table and the local Q-table;
• based on said comparison, select to use after the certain time interval either the aggregated Q-table or the local Q-table for making a decision to select the respective path for transmitting the PDU;
• and
• when the local Q-table is selected, determine that a predefined condition is present, and in response to determining that the predefined condition is present, transmit the local Q-table to the central entity.

A central entity comprising:

• at least one processor;
• and
• at least one memory storing instructions that, when executed by the at least one processor, cause the central entity located in a network at least to:
• receive information that a user device enters the network;
• transmit an aggregated Q-table to the user device;
• receive and store one or more local Q-tables, said local Q-table being received from a respective user device;
• determine that a predefined triggering condition is present;
• in response to determining that the predefined triggering condition is present, apply a federated learning based on the one or more local Q-tables;
• and
• in response to applying the federated learning, generate an updated aggregated Q-table and transmit the updated aggregated Q-table to the respective user devices in the network and to other user devices also located in the network.

A method performed at a user device, the method comprising:

• receiving a packet data unit, PDU, of one or more incoming PDUs to be transmitted by the user device, via a respective path, to one amongst a master node and at least one secondary node;
• receiving, once the user device has entered a network, an aggregated Q-table from a central entity of the network, the aggregated Q-table including rewards and being shared with the user device and with other user devices also located in the network;
• performing a local exploration during a certain time interval using the aggregated Q-table comprising one or more rewards, and generating, in response to the local exploration, a local Q-table, the local Q-table comprising rewards;
• comparing the respective rewards from the aggregated Q-table and the local Q-table;
• based on said comparison, selecting to use after the certain time interval either the aggregated Q-table or the local Q-table for making a decision to select the respective path for transmitting the PDU;
• and
• when the local Q-table is selected, determining that a predefined condition is present, and in response to determining that the predefined condition is present, transmitting the local Q-table to the central entity.