# US 2026/0089752 A1 · Hierarchical reinforcement learning for next generation of multi-ap coordinated spatial reuse

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0089752 A1
- Document ID: US20260089752A1
- Status: Published application
- Kind: A1
- Application number: 19/307,529
- Filed: 2025-08-22
- Published: 2026-03-26
- Earliest claimed: 2024-09-20
- Assignees: Sony Group Corporation; Sony Corporation of America
- Applicants: Sony Group Corporation; Sony Corporation of America
- Inventors: Ziru Chen; Salvatore Talarico; Qing Xia
- Web page: https://patentgrep.com/patent/US20260089752A1

## CPC

- H04W 74/04: Scheduled access (hybrid access H04W74/02)
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- H04W 24/02: Arrangements for optimising operational condition
- H04W 84/12: WLAN [Wireless Local Area Networks]
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Multiple Access Point Coordination (MAPC) is directed to enhance WiFi performance by enabling a set of Access Points (APs) to coordinate with each other through advanced coordinating schemes to reduce inter-AP contention and congestion. A process framework is described which facilitates coordination across multiple-APs using Coordinated Spatial Reuse (C-SR), that reciprocally adjust their scheduling strategy, power control and link adaptation to meet specific Quality of Service (QoS) requirements. A two-layer Multi-Armed Bandit (MAB) approach is described, which also preserves fair use of resources across all nodes. The validity of this approach is confirmed by system level simulations, which validate the improved efficiency in terms of sum-throughput, as well as enhanced fairness.

## Claims

### Claim 1

A station apparatus for communication in a wireless network, the apparatus comprising:
(a) at least one modem coupled to at least one radio-frequency (RF) circuit, with each RF circuit connected to one or multiple antennas;
(b) wherein said station (STA) is configured as a separate STA or as a STA within a multiple-link device (MLD);
(c) a processor of said STA;
(d) a non-transitory memory storing instructions executable by the processor for wirelessly communicating with other STAs on a IEEE 802.11 wireless local area network (WLAN); and
(e) wherein said instructions, when executed by the processor, perform steps of a wireless communications protocol for allowing an increased density of access points (APs) within the WLAN, or portion of the WLAN, using hierarchical reinforcement learning for multiple-AP (MAP) coordinated spatial reuse (CSR), comprising:
(i) wherein said STA operates in the wireless communications protocol as an access point (AP) STA configured for communicating with other STAs, including both AP STAs and non-AP STAS;
(ii) wherein said AP serves as a transmission initiator, as a sharing AP, configured for establishing coordination within a set of APs, wherein each of the other APs on this portion of the WLAN serve as followers, as shared APs;
(iii) performing multiple access point coordination (MAPC) in which the said AP, and other APs within the WLAN or portion of the WLAN, operating with this wireless communications protocol, coordinate with each other to negotiate use of radio resources in establishing a transmit power and a modulation and coding scheme (MCS) for each AP associated to each transmission link, for reducing inter-AP contention and congestion, and toward allowing Quality-of-Service (QoS) requirements to be met;
(iv) wherein said modulation and coding scheme (MCS) is determined by a two layer multiple-armed bandit (MAB) process in jointly tuning transmissions;
(v) determining coordinated-spatial reuse (C-SR) to jointly address a trade off between sum data rate and fairness using an optimization objective function comprising multiple layer reinforcement learning (RL) in a Markov decision process (MDP) characterized by a set of states S, actions A, a reward function R, and transition probabilities Pt, in which a next state of a state space, is determined solely by a current state and action of an action space, to be taken without dependence on past states, and wherein said reward function is evaluated over multiple time periods toward obtaining long-term fairness;
(vi) wherein the state space is determined by network topology and the action space having adjustable parameters comprising transmission power P, association selection A, MCS M, and QoS requirements Q, with these joint trade offs performed prior to each transmit opportunity (TXOP); and
(vii) whereby communication resources are fairly coordinated across an increased density of AP stations while controlling operations of followers on this portion of the WLAN.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein selection of said sharing AP is configured for equitable sharing of communication resources between multiple APs.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein said equitable sharing of communication resources comprises determining which AP is to be the sharing AP based on a round-robin selecting process.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein coordinating transmission scheduling and transmit power comprises using specific resource units (RUs) for each associated STA.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein both sum data rate and fairness are jointly considered in an optimization objective function.

### Claim 6 (depends on 1)

The apparatus of claim 1, further comprising consideration of long-term fairness in reward functions evaluated with respect to time.

### Claim 7 (depends on 6)

The apparatus of claim 6, further comprising executing a hierarchical MAB simulation approach in evaluating said reward functions, comprising:
adjusting the target quality-of-service (QoS) parameters in an outer layer of the MAB simulation, between the sharing AP and its associated STA, for every Touter time slot, while balancing data rate maximization and fairness;
performing an inner layer of the MAB simulation, operating at each single time slot, to fine-tune transmission power (P), association selection (A), and MCS, based on adjusted target QoS; and
feeding back performance results from the inner and outer MAB simulations back to inform and refine subsequent coordination decisions.

### Claim 8 (depends on 7)

The apparatus of claim 7:
wherein for each time slot, the inner layer of the MAB simulation randomly selects a sharing AP and one of its associated STAs for downlink transmission;
selecting a subset of shared APs based on the context of chosen AP and STA, after which for each AP in the subset, an associated STA along with its transmission power and MCS are tested through simulations to maximize data rate, with the resulting performance metrics being fed back to guide future decisions.

### Claim 9 (depends on 8)

The apparatus of claim 8, further comprising introducing a noise term into said hierarchical MAB simulations to encourage exploration during training, wherein the noise term is added, then sampled from a Gaussian distribution to balance between trying new actions and using known strategies.

### Claim 10 (depends on 1)

The apparatus of claim 1, further comprising said AP and the other APs, within the WLAN or portion of the WLAN, under this wireless communications protocol receive communications from a central controller that is responsible for coordinating all coordinating APs and performing associated resource management among these APs.

### Claim 11 (depends on 10)

The apparatus of claim 10:
wherein said AP and coordinating APs, within the WLAN or portion of the WLAN, follow the instructions provided by the central controller in their scheduling and/or resource allocation, in response to the central controller communicating several elements of control information to said coordinating APs; and
wherein the control information comprises sharing/shared AP information, STA associations, transmission power, MCS selection, and quality-of-service (QoS) requirements for all links and to all APs, whereby all APs within the WLAN or portion of the WLAN are made aware of their individual scheduling and resource management, prior to transmitting downlink signals during a transmit opportunity (TXOP).

### Claim 12

A station apparatus for communication in a wireless network, the apparatus comprising:
(a) at least one modem coupled to at least one radio-frequency (RF) circuit, with each RF circuit connected to one or multiple antennas;
(b) wherein said station (STA) is configured as a separate STA or as a STA within a multiple-link device (MLD);
(c) a processor of said STA;
(d) a non-transitory memory storing instructions executable by the processor for wirelessly communicating with other STAs on a IEEE 802.11 wireless local area network (WLAN); and
(e) wherein said instructions, when executed by the processor, perform steps of a wireless communications protocol for allowing an increased density of access points (APs) within the WLAN, or portion of the WLAN, using hierarchical reinforcement learning for multiple-AP (MAP) coordinated spatial reuse (CSR), comprising:
(i) wherein said STA operates in the wireless communications protocol as an access point (AP) STA configured for communicating with other STAs, including both AP STAs and non-AP STAS;
(ii) wherein said AP serves as a transmission initiator, as a sharing AP, configured for establishing coordination within a set of APs, wherein each of the other APs on this portion of the WLAN serve as followers, as shared APs;
(iii) performing multiple access point coordination (MAPC) in which the said AP, and other APs within the WLAN or portion of the WLAN, operating with this wireless communications protocol, coordinate with each other to negotiate use of radio resources and specific resource units (RUs) in establishing a transmit power and a modulation and coding scheme (MCS) for each AP associated to each transmission link, for reducing inter-AP contention and congestion, and toward allowing Quality-of-Service (QoS) requirements to be met; and
(iv) determining coordinated-spatial reuse (C-SR) to jointly address a trade off between sum data rate and fairness using an optimization objective function comprising multiple layer reinforcement learning (RL) in a Markov decision process (MDP) characterized by a set of states S, actions A, a reward function R, and transition probabilities Pt, in which a next state of a state space, is determined solely by a current state and action of an action space, to be taken without dependence on past states, and wherein said reward function is evaluated over multiple time periods toward obtaining long-term fairness;
(v) wherein the state space is determined by network topology and the action space having adjustable parameters comprising transmission power P, association selection A, MCS M, and QoS requirements Q, with these joint trade offs performed prior to each transmit opportunity (TXOP); and
(vi) whereby communication resources are fairly coordinated across an increased density of AP stations while controlling operations of followers on this portion of the WLAN.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein selection of said sharing AP is configured for equitable sharing of communication resources between multiple APs based on a round-robin selecting process.

### Claim 14 (depends on 12)

The apparatus of claim 12, wherein both sum data rate and fairness are jointly considered in an optimization objective function.

### Claim 15 (depends on 12)

The apparatus of claim 12, further comprising consideration of long-term fairness in reward functions evaluated with respect to time.

### Claim 16 (depends on 12)

The apparatus of claim 12, further comprising executing a hierarchical MAB simulation approach in evaluating said reward functions, comprising:
adjusting the target quality-of-service (QoS) parameters in an outer layer of the MAB simulation, between the sharing AP and its associated STA, for every Touter time slot, while balancing data rate maximization and fairness;
performing an inner layer of the MAB simulation, operating at each single time slot, to fine-tune transmission power (P), association selection (A), and MCS, based on adjusted target QoS; and
feeding back performance results from the inner and outer MAB simulations back to inform and refine subsequent coordination decisions.

### Claim 17 (depends on 16)

The apparatus of claim 16:
wherein for each time slot, the inner layer of the MAB simulation randomly selects a sharing AP and one of its associated STAs for downlink transmission;
selecting a subset of shared APs based on the context of chosen AP and STA, after which for each AP in the subset, an associated STA along with its transmission power and MCS are tested through simulations to maximize data rate, with the resulting performance metrics being fed back to guide future decisions.

### Claim 18 (depends on 17)

The apparatus of claim 17, further comprising introducing a noise term into said hierarchical MAB simulations to encourage exploration during training, wherein the noise term is added, then sampled from a Gaussian distribution to balance between trying new actions and using known strategies.

### Claim 19 (depends on 12)

The apparatus of claim 12, further comprising said AP and the other APs, within the WLAN or portion of the WLAN, under this wireless communications protocol receive communications from a central controller that is responsible for coordinating all coordinating APs and performing associated resource management among these APs.

### Claim 20

A method of performing communications in a wireless network, comprising:
(a) performing wireless communications between STAs on a IEEE 802.11 wireless local area network (WLAN), following steps of a wireless communications protocol for allowing an increased density of access points (APs) within the WLAN, or portion of the WLAN, using hierarchical reinforcement learning for multiple-AP (MAP) coordinated-spatial reuse (C-SR), comprising:
(i) wherein one AP serves as a transmission initiator, which operates as a sharing AP, configured for establishing coordination within a set of APs, wherein each of the other APs on this portion of the WLAN serve as followers, operating as shared APs;
(ii) performing multiple access point coordination (MAPC) in which the said AP, and other APs within the WLAN or portion of the WLAN, operating with this wireless communications protocol, coordinate with each other to negotiate use of radio resources in establishing a transmit power and a modulation and coding scheme (MCS) for each AP associated to each transmission link, for reducing inter-AP contention and congestion, and toward allowing Quality-of-Service (QoS) requirements to be met; and
(iii) wherein said modulation and coding scheme (MCS) is determined by a two layer multiple-armed bandit (MAB) process in jointly tuning transmissions;
(iv) determining coordinated-spatial reuse (C-SR) to jointly address a trade off between sum data rate and fairness using an optimization objective function comprising multiple layer reinforcement learning (RL) in a Markov decision process (MDP) characterized by a set of states S, actions A, a reward function R, and transition probabilities Pt, in which a next state of a state space, is determined solely by a current state and action of an action space, to be taken without dependence on past states, and wherein said reward function is evaluated over multiple time periods toward obtaining long-term fairness;
(v) wherein the state space is determined by network topology and the action space having adjustable parameters comprising transmission power P, association selection A, MCS M, and QoS requirements Q, with these joint trade offs performed prior to each transmit opportunity (TXOP); and
(vi) whereby communication resources are fairly coordinated across an increased density of AP stations while controlling operations of followers on this portion of the WLAN.