Energy-aware routing based on reinforcement learning

A non-transitory computer-readable medium having stored thereon computer-executable, instructions that, when executed by a processing circuitry, cause the processing circuitry to execute operations, the operations comprising:

• acquiring observations characterising a current state of the plurality of IAB nodes, wherein the observations comprise:
• routing information for routing packets in the network, energy information indicative of an energy performance of each of the plurality of IAB nodes, and traffic information indicative of data traffic performance of each of the plurality of IAB nodes;
• and
• performing the following steps iteratively until a termination condition is met:
• determining an action to be performed from a predetermined set of actions using a selection policy and based on latest acquired observations, wherein the predetermined set of actions include adding an entry to the routing information and removing an entry to the routing information, wherein an entry is indicative of how packets are to be routed with respect to an IAB node of the plurality of IAB nodes;
• executing the action by initiating update of the routing information based on the determined action;
• acquiring observations characterising an updated state of the plurality of IAB nodes subsequent to execution of the action;
• determining a reward for the determined action, based on the updated state of the plurality of IAB nodes;
• storing an experience set including the determined action, the observations characterising the state of the plurality of IAB nodes prior to execution of the determined action, the observations characterising the state of the plurality of IAB nodes subsequent to execution of the determined action, and the determined reward;
• and
• training the reinforcement learning system to maximise reward with respect to an optimisation objective, using the one or more stored experience sets in the buffer.

An Integrated Access and Backhaul (IAB) donor node configured to:

• acquire observations characterising a current state of the plurality of IAB nodes, wherein the observations comprise:
• routing information for routing packets in the network, energy information indicative of an energy performance of each of the plurality of IAB nodes, and traffic information indicative of data traffic performance of each of the plurality of IAB nodes;
• and
• perform the following steps iteratively until a termination condition is met:
• determine an action to be performed from a predetermined set of actions using a selection policy and based on latest acquired observations, wherein the predetermined set of actions include adding an entry to the routing information and removing an entry to the routing information, wherein an entry is indicative of how packets are to be routed with respect to an IAB node of the plurality of IAB nodes;
• execute the action by initiating update of the routing information based on the determined action;
• acquire observations characterising an updated state of the plurality of IAB nodes subsequent to execution of the action;
• determine a reward for the determined action, based on the updated state of the plurality of IAB nodes;
• store an experience set including the determined action, the observations characterising the state of the plurality of IAB nodes prior to execution of the determined action, the observations characterising the state of the plurality of IAB nodes subsequent to execution of the determined action, and the determined reward;
• and
• train the reinforcement learning system to maximise reward with respect to an optimisation objective, using the one or more stored experience sets in the buffer.