US 12,764,038 B2 · Data-model dual-driven congestion control method for data center networks

Abstract

Disclosed in the present disclosure is a data-model dual-driven congestion control method for data center networks, which combines adaptability of deep reinforcement learning with fast responsiveness of conventional model mapping to address the problems of slow response speed and insufficient adaptability in a dynamic network environment in the prior art. The method employs a layered architecture, which includes an optimization engine module, a regulation control module, and a statistical information module. The optimization engine module learns network state data based on a deep reinforcement learning algorithm and dynamically optimizes a congestion control policy. The regulation control module quickly executes the policy by using a model mapping mechanism to regulate transmission rate in real time. The statistical information module continuously collects network state information to ensure that a system adapts to complex traffic changes.

A data-model dual-driven congestion control method for data center networks, comprising:

S1, constructing a system framework comprising an optimization engine module, a statistical information module, and a rate regulation control module, wherein the optimization engine module optimizes a congestion control sub-policy in real time by using an optimization algorithm, the statistical information module is responsible for collecting network operation data and providing support for a decision process and system monitoring of an optimization engine, the optimization engine module and the statistical information module jointly form an RL agent, and the regulation control module is configured to execute the sub-policy generated by the RL agent;
S2, constructing a parameterized rate regulation model for rates at a network end side, specifically comprising:
S21, calculating a convergence result by using a network parameter and queuing delay to serve as a reference for rate adjustment, wherein a calculation formula is as follows:
R
REF
=
1
δ
×
d
q
d
q
=
rtt
new
-
min
⁢
{
rtt
new
,
rtt
min
}
wherein δ is a regulation parameter and 0.5≤δ≤1.5, and the parameter is used to indicate a state of total network bandwidth and transmission requirement, and dq is the queuing delay and is a result obtained after serialization delay and minimum RTT processing are performed on an RTT feedback;
a rate update cycle needs to be adaptively adjusted based on network requirements, the parameter δ reflects the network requirements and is determined by link bandwidth and the number of connections, and the update cycle is designed as:
T
a
=
δ
×
K
T
++
wherein Ta is an adaptive byte counter threshold, K represents the link bandwidth, and T++ represents an update of a byte counter;
to improve convergence speed and fairness, the adaptive byte counter is designed to be independent of transmission rates, that is:
BC
a
=
γ
×
R
C
BC
++
wherein BCa is an adaptive byte counter threshold, γ is a regulation parameter and 0.5≤γ≤1.5, and Rc is a current transmission rate;
S22, designing a rate decrease mechanism for the regulation model, specifically as follows:
R
T
=
{
R
C
,
R
C
<
8
⁢
R
REF
R
C
μ
1
,
otherwise
R
C
=
{
R
C
(
1
-
α
2
)
,
R
C
<
8
⁢
R
REF
R
C
μ
2
,
otherwise
wherein RT represents a target transmission rate that needs to be reached through adjustment, Rc represents a current end-side transmission rate, parameters μ1 and μ2 are regulation parameters, and 2<μ1≤μ2≤4;
the parameters are used to control aggressiveness of rate decrease when congestion occurs;
if CNP is not received within a specified time, α needs to be updated, that is:
α
=
(
1
-
g
)
⁢
α
+
g
wherein α is a regulation factor, and g is a fixed value for adjusting α;
S23, designing a rate increase mechanism for the regulation model, wherein the rate increase comprises a fast recovery phase, an adaptive increase phase, and a hyper increase phase;
the fast recovery phase:
when both a timer and the byte counter are smaller than a threshold, that is, T<F and BC<F, a fast recovery operation is performed, wherein μ3 is a regulation parameter and 6≤μ3≤10, and the design is as follows:
R
T
=
{
R
T
,
R
C
<
3
⁢
R
REF
R
T
-
R
C
μ
3
,
otherwise
R
C
=
R
C
+
R
T
2
the adaptive increase phase:
when the timer or the byte counter is greater than the threshold, that is, T≥F or BC≥F, an adaptive additive increase operation is performed, wherein an adaptive increase step size is related to the current transmission rate and the link bandwidth, but not to the number of carried flows, and the design is as follows:
R
T
=
{
R
T
+
min
⁢
(
0.1
R
C
,
β
*
R
L
)
,
R
C
<
3
⁢
R
REF
R
T
-
R
C
μ
3
,
otherwise
R
C
=
R
C
+
R
T
2
wherein RL is link capacity, β and μ3 are regulation parameters, and 0<β<1;
the hyper increase phase:
when both the timer and the byte counter are greater than the threshold, that is, T≥F and BC≥F, a hyper increase operation is performed, and the design is as follows:
R
T
=
{
R
T
+
iR
HAI
,
R
C
<
3
⁢
R
REF
R
T
-
R
C
μ
3
,
otherwise
R
C
=
R
C
+
R
T
2
wherein i=min (BC,T−F+1), RHAI is a fixed value 100 Mbps, and μ3 is a regulation parameter;
S24, setting a corresponding control parameter in the parameterized rate regulation model for rates at the network end side to be dynamically adjustable after an overall design of a regulation control model is determined, that is, defining θ as:
θ
:=
{
δ
,
γ
,
β
,
μ
1
,
μ
2
,
μ
3
}
S3, optimizing the dynamically adjustable parameter in S23 based on a DRL algorithm and formulates a congestion control problem in a network as a Markov decision model, wherein the model comprises setting state variables and a reward function, and for description of network conditions, a switch buffer occupancy rate Occ is introduced to describe congestion degree, and the indicator is represented by proportion of data packets marked with an ECN;
S4, designing and solving the state and the reward function in S3 based on the DRL algorithm, and selecting proximal policy optimization PPO as an RL controller, wherein the PPO algorithm performs policy optimization based on a Markov decision process.

249 more elements

Data-model dual-driven congestion control method for data center networks

Life of this patent

Data-model dual-driven congestion control method for data center networks

Life of this patent

Abstract

Claims5