Hierarchical reinforcement learning for next generation of multi-ap coordinated spatial reuse

A station apparatus for communication in a wireless network, the apparatus comprising:

• (a) at least one modem coupled to at least one radio-frequency (RF) circuit, with each RF circuit connected to one or multiple antennas;
• (b) wherein said station (STA) is configured as a separate STA or as a STA within a multiple-link device (MLD);
• (c) a processor of said STA;
• (d) a non-transitory memory storing instructions executable by the processor for wirelessly communicating with other STAs on a IEEE 802.11 wireless local area network (WLAN);
• and
• (e) wherein said instructions, when executed by the processor, perform steps of a wireless communications protocol for allowing an increased density of access points (APs) within the WLAN, or portion of the WLAN, using hierarchical reinforcement learning for multiple-AP (MAP) coordinated spatial reuse (CSR), comprising:
• (i) wherein said STA operates in the wireless communications protocol as an access point (AP) STA configured for communicating with other STAs, including both AP STAs and non-AP STAS;
• (ii) wherein said AP serves as a transmission initiator, as a sharing AP, configured for establishing coordination within a set of APs, wherein each of the other APs on this portion of the WLAN serve as followers, as shared APs;
• (iii) performing multiple access point coordination (MAPC) in which the said AP, and other APs within the WLAN or portion of the WLAN, operating with this wireless communications protocol, coordinate with each other to negotiate use of radio resources and specific resource units (RUs) in establishing a transmit power and a modulation and coding scheme (MCS) for each AP associated to each transmission link, for reducing inter-AP contention and congestion, and toward allowing Quality-of-Service (QoS) requirements to be met;
• and
• (iv) determining coordinated-spatial reuse (C-SR) to jointly address a trade off between sum data rate and fairness using an optimization objective function comprising multiple layer reinforcement learning (RL) in a Markov decision process (MDP) characterized by a set of states S, actions A, a reward function R, and transition probabilities Pt, in which a next state of a state space, is determined solely by a current state and action of an action space, to be taken without dependence on past states, and wherein said reward function is evaluated over multiple time periods toward obtaining long-term fairness;
• (v) wherein the state space is determined by network topology and the action space having adjustable parameters comprising transmission power P, association selection A, MCS M, and QoS requirements Q, with these joint trade offs performed prior to each transmit opportunity (TXOP);
• and
• (vi) whereby communication resources are fairly coordinated across an increased density of AP stations while controlling operations of followers on this portion of the WLAN.

A method of performing communications in a wireless network, comprising:

• (a) performing wireless communications between STAs on a IEEE 802.11 wireless local area network (WLAN), following steps of a wireless communications protocol for allowing an increased density of access points (APs) within the WLAN, or portion of the WLAN, using hierarchical reinforcement learning for multiple-AP (MAP) coordinated-spatial reuse (C-SR), comprising:
• (i) wherein one AP serves as a transmission initiator, which operates as a sharing AP, configured for establishing coordination within a set of APs, wherein each of the other APs on this portion of the WLAN serve as followers, operating as shared APs;
• (ii) performing multiple access point coordination (MAPC) in which the said AP, and other APs within the WLAN or portion of the WLAN, operating with this wireless communications protocol, coordinate with each other to negotiate use of radio resources in establishing a transmit power and a modulation and coding scheme (MCS) for each AP associated to each transmission link, for reducing inter-AP contention and congestion, and toward allowing Quality-of-Service (QoS) requirements to be met;
• and
• (iii) wherein said modulation and coding scheme (MCS) is determined by a two layer multiple-armed bandit (MAB) process in jointly tuning transmissions;
• (iv) determining coordinated-spatial reuse (C-SR) to jointly address a trade off between sum data rate and fairness using an optimization objective function comprising multiple layer reinforcement learning (RL) in a Markov decision process (MDP) characterized by a set of states S, actions A, a reward function R, and transition probabilities Pt, in which a next state of a state space, is determined solely by a current state and action of an action space, to be taken without dependence on past states, and wherein said reward function is evaluated over multiple time periods toward obtaining long-term fairness;
• (v) wherein the state space is determined by network topology and the action space having adjustable parameters comprising transmission power P, association selection A, MCS M, and QoS requirements Q, with these joint trade offs performed prior to each transmit opportunity (TXOP);
• and
• (vi) whereby communication resources are fairly coordinated across an increased density of AP stations while controlling operations of followers on this portion of the WLAN.