Back to results
US 12,615,549 B2Granted
Traffic load management
AssigneeTelefonaktiebolaget LM Ericsson (pub)
InventorsAlessandro Previti, Alexandros Nikou, Aneta Vulgarakis Feljan, Athanasios Karapantelakis
Life of this patent
Estimating the term…
Abstract
A reinforcement learning (RL) process is used to allocate UL/DL resources and is also used to offload traffic from a first access point (e.g., macro access point) to a second access point (e.g., a micro access point) when the load on the first access point is too high. The RL process is able to handle the dynamic nature of UL/DL imbalances and is therefore able to maximize usage of resources without compromising quality of service.
Claims16
A reinforcement learning (RL) method for traffic load management, the method being performed by a RL agent of a first access point and comprising:
- for each action included in a group of actions, storing an accumulated reward value associated with the action, thereby storing a set of accumulated reward values where each accumulated reward value include in the set is associated with a different one of the actions;
- for each action included in the group of actions, calculating a first action value for the action using the stored accumulated reward value for the action as one input to the calculation;
- based on the calculated action values, selecting an action from the group of actions;
- performing the selected action;
- after performing the selected action, determining an uplink (UL) throughput status value for a radio unit of the first access point and determining a downlink (DL) throughput status value for the radio unit, wherein the radio unit has an UL throughput capacity and a DL throughput capacity;
- determining a reward value based on the determined UL and DL throughput status values;
- updating the stored accumulated reward value for the selected action based on the determined reward value;
- and
- initiating the handover of a user equipment, UE, from the first access point to a second access point if the determined reward value is equal to a predetermined value;
- wherein the determined UL throughput status value indicates either i) that the UL throughput capacity is underused or ii) that the UL throughput capacity is overused, and the determined DL throughput status value indicates either i) that the DL throughput capacity is underused or ii) that the DL throughput capacity is overused;
- wherein determining the reward value based on the determined UL and DL throughput status values comprises:
- determining whether the UL throughput status value indicates that the UL throughput capacity is overused;
- and
- determining whether the DL throughput status value indicates that the DL throughput capacity is overused;
- wherein determining the reward value further comprises:
- setting the reward value equal to R as a result of either:
- i) determining that the UL throughput status value indicates that the UL throughput capacity is underused and the DL throughput status value indicates that the DL throughput capacity is overused or ii) determining that the UL throughput status value indicates that the UL throughput capacity is overused and the DL throughput status value indicates that the DL throughput capacity is underused, where
- R=1−((UL_T+DL_T)/(MAX_UL_T+MAXDL_T)),
- UL_T is a measured UL throughput value,
- DL_T is a measured DL throughput value,
- MAX_UL_T is a value specifying the radio unit's UL throughput capacity, and
- MAX_DL_T is a value specifying the radio unit's DL throughput capacity.
19 more elements
An access point, the access point comprising:
- processing circuitry;
- and
- a memory, the memory containing instructions executable by the processing circuitry, wherein the access point is configured to perform a process comprising:
- for each action included in a group of actions, storing an accumulated reward value associated with the action, thereby storing a set of accumulated reward values where each accumulated reward value include in the set is associated with a different one of the actions;
- for each action included in the group of actions, calculating a first action value for the action using the stored accumulated reward value for the action as one input to the calculation;
- based on the calculated action values, selecting an action from the group of actions;
- performing the selected action;
- after performing the selected action, determining an uplink (UL) throughput status value for a radio unit of the first access point and determining a downlink (DL) throughput status value for the radio unit, wherein the radio unit has an UL throughput capacity and a DL throughput capacity;
- determining a reward value based on the determined UL and DL throughput status values;
- updating the stored accumulated reward value for the selected action based on the determined reward value;
- and
- initiating the handover of a user equipment, UE, from the first access point to a second access point if the determined reward value is equal to a predetermined value;
- wherein the determined UL throughput status value indicates either i) that the UL throughput capacity is underused or ii) that the UL throughput capacity is overused, and the determined DL throughput status value indicates either i) that the DL throughput capacity is underused or ii) that the DL throughput capacity is overused;
- wherein determining the reward value based on the determined UL and DL throughput status values comprises:
- determining whether the UL throughput status value indicates that the UL throughput capacity is overused;
- and
- determining whether the DL throughput status value indicates that the DL throughput capacity is overused;
- wherein determining the reward value further comprises:
- setting the reward value equal to R as a result of either:
- i) determining that the UL throughput status value indicates that the UL throughput capacity is underused and the DL throughput status value indicates that the DL throughput capacity is overused or ii) determining that the UL throughput status value indicates that the UL throughput capacity is overused and the DL throughput status value indicates that the DL throughput capacity is underused, where
- R=1−((UL_T+DL_T)/(MAX_UL_T+MAXDL_T)),
- UL_T is a measured UL throughput value,
- DL_T is a measured DL throughput value,
- MAX_UL_T is a value specifying the radio unit's UL throughput capacity, and
- MAX_DL_T is a value specifying the radio unit's DL throughput capacity.
22 more elements
Dependencies are inferred from the claim text.