US 2023/0156794 A1 · UNSLOTTED CSMACA OPTIMIZATION METHOD AND DEVICES IN WI-SUN USING REINFORCEMENT LEARNING

Abstract

An unslotted CSMA/CA optimization device includes a variable initializing unit performing variable initialization used in an algorithm for unslotted CSMA/CA optimization; an exploration and exploitation selecting unit determining exploration/exploitation using an epsilon greedy algorithm; an action selecting unit selecting an action having the best Q-value, among actions, when exploitation is selected, and randomly selecting an action when exploration is selected; a channel information collecting unit executing backoff when an action (backoff time) is selected, repeatedly executing CCA during the backoff time, and counting the number of times a channel is idle and the number of times the channel is busy; a success rewarding unit transmitting a packet when the channel is idle and rewarding success when acknowledge (Ack) is received; and a Q-table updating unit checking the received reward and updating a Q-table based on an action, a state, and a reward.

Claims17

Dependent claims

An unslotted CSMA/CA optimization device in a Wi-SUN using reinforcement learning, the unslotted CSMA/CA optimization device comprising:

a variable initializing unit performing variable initialization used in an algorithm for unslotted CSMA/CA optimization;
an exploration and exploitation selecting unit determining exploration/exploitation using an epsilon greedy algorithm;
an action selecting unit selecting an action having the best Q-value, among actions, when exploitation is selected, and randomly selecting an action when exploration is selected;
a channel information collecting unit executing backoff when an action (backoff time) is selected, repeatedly executing channel clear access (CCA) during the backoff time, and counting the number of times a channel is idle and the number of times the channel is busy;
a success rewarding unit transmitting a packet when the channel is idle and rewarding success when acknowledge (Ack) is received;
and
a Q-table updating unit checking the received reward and updating a Q-table based on an action, a state, and a reward.

4 more elements

An unslotted CSMA/CA optimization method in a wireless smart utility network (Wi-SUN) using reinforcement learning, the unslotted CSMA/CA optimization method comprising:

a variable initializing operation of performing variable initialization used in an algorithm for unslotted CSMA/CA optimization;
an exploration and utilization selecting operation of determining exploration/exploitation using an epsilon greedy algorithm;
an action selecting operation of selecting an action having the best Q-value, among actions, when exploitation is selected, and randomly selecting an action when exploration is selected;
a channel information collecting operation of executing backoff when an action (backoff time) is selected, repeatedly executing channel clear access (CCA) during the backoff time, and counting the number of times a channel is idle and the number of times the channel is busy;
a success rewarding operation of transmitting a packet when the channel is idle and rewarding success when acknowledge (Ack) is received;
and
a Q-table updating operation of checking the received reward and updating a Q-table based on an action, a state, and a reward.

4 more elements

Dependencies are inferred from the claim text.

Unslotted csmaca optimization method and devices in wi-sun using reinforcement learning

Life of this patent

Unslotted csmaca optimization method and devices in wi-sun using reinforcement learning

Life of this patent

Abstract

Claims17