# US 11,310,159 B2 · Packet sending method and apparatus, and storage device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,310,159 B2
- Document ID: US11310159B2
- Status: Granted
- Kind: B2
- Application number: 16/942,988
- Filed: 2020-07-30
- Published: 2020-11-12
- Granted: 2022-04-19
- Earliest claimed: 2018-03-06
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Dexian Su
- Web page: https://patentgrep.com/patent/US11310159B2

## CPC

- H04L 47/11: Identifying congestion
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence)
- H04L 47/115: {using a dedicated packet}
- H04L 47/12: Avoiding congestion; Recovering from congestion
- H04L 47/25: with rate being modified by the source upon detecting a change of network conditions
- H04L 47/31: by tagging of packets, e.g. using discard eligibility [DE] bits
- H04L 49/103: using a shared central buffer; using a shared memory
- G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances}
- H04L 47/24: Traffic characterised by specific attributes, e.g. priority or QoS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A packet sending method is applied to a data transmission system and comprising: receiving, by a network interface card of a second device, a first packet that is from a first QP of a first device and that is forwarded by using a switch; obtaining, by the network interface card of the second device, an instant rate of the first QP and a maximum sending rate of the first QP; determining, by the network interface card of the second device, an adjustment rate of the first QP based on the instant rate of the first QP and the maximum sending rate of the first QP; sending, by the network interface card of the second device, a congestion notification packet to the first device, where the congestion notification packet carries the adjustment rate of the first QP.

## Claims

### Claim 1

A packet sending method implemented by a second device and comprising:
receiving, from a first device and using an RDMA over Converged Ethernet (RoCE) protocol, a first packet;
obtaining, in response to the first packet, a first instant rate of a first queue pair (QP) and a first maximum sending rate of the first QP, wherein the first instant rate represents a quantity of packets that are sent between the first QP and a first QP′ within a unit time, wherein the first QP′ is associated with the first QP, and wherein the first maximum sending rate represents a preset maximum quantity of packets that are allowed to be sent between the first QP and the first QP′ within the unit time;
identifying an adjustment rate of the first QP based on the first instant rate and the first maximum sending rate; and
sending, to the first device, a congestion notification packet comprising the adjustment rate.

### Claim 2 (depends on 1)

The packet sending method of claim 1, further comprising further identifying the adjustment rate based on a weight coefficient corresponding to a priority of the first device.

### Claim 3 (depends on 1)

The packet sending method of claim 1, further comprising further identifying the adjustment rate based on a weight coefficient corresponding to a priority of a virtual machine bound to the first QP.

### Claim 4 (depends on 1)

The packet sending method of claim 1, further comprising further identifying the adjustment rate based on a priority of the first device and a weight coefficient corresponding to a priority of a virtual machine bound to the first QP.

### Claim 5 (depends on 1)

The packet sending method of claim 1, further comprising further identifying the adjustment rate using the following formula:
S
1
=
D
⁢
A
1
/
C
1
∑
i
=
1
m
⁢
A
i
/
C
i
,
wherein S1 is the adjustment rate, D is a port rate during congestion of a switch, A1 is the first instant rate, C1 is the first maximum sending rate, Ai is an instant rate of an ithQP of m QPs, Ci is a maximum sending rate of the ithQP, m is a positive integer, and i is a positive integer ranging from 1 to m.

### Claim 6 (depends on 1)

The packet sending method of claim 1, further comprising further identifying the adjustment rate using the following formula:
S
1
=
D
⁢
B
1
⁢
A
1
/
C
1
∑
i
=
1
m
⁢
A
i
/
C
i
,
wherein S1 is the adjustment rate, D is a port rate during congestion of a switch, B1 is a weight coefficient corresponding to a priority of the first device, A1 is the first instant rate, C1 is the first maximum sending rate of the first QP, Ai is an instant rate of an ith QP of m QPs, Ci is a maximum sending rate of the ith QP, m is a positive integer, and i is a positive integer ranging from 1 to m.

### Claim 7 (depends on 1)

The packet sending method of claim 1, wherein before obtaining the first instant rate and the first maximum sending rate, the packet sending method further comprises:
reading an explicit congestion notification (ECN) flag bit of the first packet; and
identifying that the ECN flag bit indicates that network congestion occurs on a switch.

### Claim 8

A storage device comprising:
a memory configured to store instructions; and
a processor coupled to the memory and configured to execute the instructions to:
receive, from a first device and using an RDMA over Converged Ethernet (RoCE) protocol, a first packet;
obtain, in response to the first packet, a first instant rate of a first queue pair (QP) and a first maximum sending rate of the first QP, wherein the first instant rate represents a quantity of packets that are sent between the first QP and a first QP′ within a unit time, wherein the first QP′ is associated with the first QP, and wherein the first maximum sending rate represents a preset maximum quantity of packets that are allowed to be sent between the first QP and the first QP′ within the unit time;
identify an adjustment rate of the first QP based on the first instant rate and the first maximum sending rate; and
send, to the first device, a congestion notification packet comprising the adjustment rate.

### Claim 9 (depends on 8)

The storage device of claim 8, wherein the processor is further configured to further identify the adjustment rate based on a weight coefficient corresponding to a priority of the first device.

### Claim 10 (depends on 8)

The storage device of claim 8, wherein the processor is further configured to further identify the adjustment rate based on a weight coefficient corresponding to a priority of a virtual machine bound to the first QP.

### Claim 11 (depends on 8)

The storage device of claim 8, wherein the processor is further configured to further identify the adjustment rate based on a priority of the first device and a weight coefficient corresponding to a priority of a virtual machine bound to the first QP.

### Claim 12 (depends on 8)

The storage device of claim 8, wherein the processor is further configured to further identify the adjustment rate using the following formula:
S
1
=
D
⁢
A
1
/
C
1
∑
i
=
1
m
⁢
A
i
/
C
i
,
wherein S1 is the adjustment rate, D is a port rate during congestion of a switch, A1 is the first instant rate, C1 is the first maximum sending rate, Ai is an instant rate of an ith QP of m QPs, Ci is a maximum sending rate of the ithQP, m is a positive integer, and i is a positive integer ranging from 1 to m.

### Claim 13 (depends on 8)

The storage device of claim 8, wherein the processor is further configured to further identify the adjustment rate using the following formula:
S
1
=
D
⁢
B
1
⁢
A
1
/
C
1
∑
i
=
1
m
⁢
A
i
/
C
i
,
wherein S1 is the adjustment rate, D is a port rate during congestion of a switch, B1 is a weight coefficient corresponding to a priority of the first device, A1 is the first instant rate, C1 is the first maximum sending rate of the first QP, Ai is an instant rate of an ith QP of m QPs, Ci is a maximum sending rate of the ith QP of the m QPs, m is a positive integer, and i is a positive integer ranging from 1 to m.

### Claim 14 (depends on 8)

The storage device of claim 8, wherein before obtaining the first instant rate and the first maximum sending rate, the processor is further configured to:
read an explicit congestion notification (ECN) flag bit of the first packet; and
identify that the ECN flag bit indicates that network congestion occurs on a switch.

### Claim 15

A computer program product comprising instructions for storage on a non-transitory medium and that, when executed by a processor, cause a storage device to:
receive, from a first device and using an RDMA over Converged Ethernet (RoCE) protocol, a first packet;
obtain, in response to the first packet, a first instant rate of a first queue pair (QP) and a first maximum sending rate of the first QP, wherein the first instant rate represents a quantity of packets that are sent between the first QP and a first QP′ within a unit time, wherein the first QP′ is associated with the first QP, and wherein the first maximum sending rate represents a preset maximum quantity of packets that are allowed to be sent between the first QP and the first QP′ within the unit time;
identify an adjustment rate of the first QP based on the first instant rate and the first maximum sending rate; and
send, to the first device, a congestion notification packet comprising the adjustment rate.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the instructions further cause the storage device to further identify the adjustment rate based on a weight coefficient corresponding to a priority of the first device.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein the instructions further cause the storage device to further identifying the adjustment rate based on a weight coefficient corresponding to a priority of a virtual machine bound to the first QP.

### Claim 18 (depends on 15)

The computer program product of claim 15, wherein the instructions further cause the storage device to further identify the adjustment rate based on a priority of the first device, and a weight coefficient corresponding to a priority of a virtual machine bound to the first QP.

### Claim 19 (depends on 15)

The computer program product of claim 15, wherein the instructions further cause the storage device to further identify the adjustment rate using the following formula:
S
1
=
D
⁢
A
1
/
C
1
∑
i
=
1
m
⁢
A
i
/
C
i
,
wherein S1 is the adjustment rate, D is a port rate during congestion of a switch, A1 is the first instant rate, C1 is the first maximum sending rate, Ai is an instant rate of an ith QP of m QPs, Ci is a maximum sending rate of the ithQP, m is a positive integer, and i is a positive integer ranging from 1 to m.

### Claim 20 (depends on 15)

The computer program product of claim 15, wherein the instructions further cause the storage device to further identify the adjustment rate using the following formula:
S
1
=
D
⁢
B
1
⁢
A
1
/
C
1
∑
i
=
1
m
⁢
A
i
/
C
i
,
wherein S1 is the adjustment rate, D is a port rate during congestion of a switch, B1 is a weight coefficient corresponding to a priority of the first device, A1 is the first instant rate, C1 is the first maximum sending rate, Ai is an instant rate of an ith QP of m QPs, Ci is a maximum sending rate of the ithQP, m is a positive integer, and i is a positive integer ranging from 1 to m.