# US 2021/0173573 A1 · Method for assuring quality of service in distributed storage system, control node, and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0173573 A1
- Document ID: US20210173573A1
- Status: Published application
- Kind: A1
- Application number: 17/175,331
- Filed: 2021-02-12
- Published: 2021-06-10
- Earliest claimed: 2018-08-16
- Assignees: Huawei Technologies Co., Ltd.; William Marsh Rice University
- Applicants: Huawei Technologies Co., Ltd.; William Marsh Rice University
- Inventors: Si YU; Junhui GONG; Peter VARMAN; Yuhan PENG
- Web page: https://patentgrep.com/patent/US20210173573A1

## CPC

- G06F 3/0631: {by allocating resources to storage systems}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G06F 3/061: {Improving I/O performance}
- G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling}
- G06F 3/0673: {Single storage device}
- H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

The present disclosure discloses a method for assuring quality of service in a storage system, where a control node calculates, based on a quantity of remaining I/O requests of a target storage node in a unit time, a quantity of I/O requests required by a storage resource to reach a lower assurance limit in the unit time, and a quantity of I/O requests need to be processed by the target storage node for the storage resource in the unit time, a lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time; allocates, based on the lower limit quantity of I/O requests, a lower limit quantity of tokens of the storage resource on the target storage node in the unit time to the storage resource; and sends the lower limit quantity of tokens to the target storage node.

## Claims

### Claim 1

A method for assuring quality of service in a storage system, comprising:
obtaining, by a control node based on a quantity of remaining input/output (I/O) requests of a target storage node in a unit time, a quantity of I/O requests required by a storage resource to reach a lower assurance limit in the unit time, and a quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time, a lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, wherein the storage system comprises N storage nodes that provide the storage resource, the target storage node is one of the N storage nodes, and N is a positive integer;
allocating, by the control node, a lower limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time to the storage resource based on the lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time; and
sending, by the control node, the lower limit quantity of tokens to the target storage node.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
obtaining, by the control node based on the quantity of remaining I/O requests of the target storage node in the unit time, a quantity of I/O requests required by the storage resource to reach an upper flow control limit in the unit time, the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time, and the lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, an upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time;
allocating, by the control node, an upper limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time to the storage resource based on the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time; and
sequentially sending, by the control node, the upper limit quantity of tokens to the target storage node.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising:
calculating, by the control node based on a maximum flow algorithm, the quantity of remaining input/output I/O requests of the target storage node in the unit time, the quantity of I/O requests required by the storage resource to reach the lower assurance limit in the unit time, and the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time, the lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time.

### Claim 4 (depends on 2)

The method according to claim 2, further comprising:
calculating, by the control node, a first value, wherein the first value is a difference between a second value and a third value, the second value is the quantity of I/O requests required by the storage resource to reach the upper flow control limit in the unit time, and the third value is a sum of lower limit quantities of I/O requests that can be processed by the N storage nodes for the storage resource in the unit time;
calculating, by the control node, a fourth value, wherein the fourth value is a difference between a fifth value and the third value, and the fifth value is the quantity of remaining I/O requests of the target storage node in the unit time;
calculating, by the control node, a sixth value, wherein the sixth value is a difference between a seventh value and the third value, and the seventh value is the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time; and
obtaining, by the control node based on the first value, the fourth value, and the sixth value, the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time.

### Claim 5 (depends on 4)

The method according to claim 4, further comprising:
allocating, by the control node, an overcommitment value that is of the storage resource on the target storage node and that is in the unit time to the storage resource according to the following formula:
the overcommitment value that is of the storage resource on the target storage node and that is in the
unit


time
=
an


eigth


value
+
(
the


second


value
-
a


ninth


value
)
×
the


seventh


value
a


tenth


value
,
wherein the eighth value is the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, the ninth value is a sum of upper limit quantities of I/O requests that can be processed by the N storage nodes for the storage resource in the unit time, and the tenth value is a sum of quantities of I/O requests that need to be processed by the N storage nodes for the storage resource in the unit time; and
allocating, by the control node, the upper limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time to the storage resource, wherein the upper limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time is a sum of the overcommitment value that is of the storage resource on the target storage node and that is in the unit time and the lower limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time.

### Claim 6 (depends on 4)

The method according to claim 4, further comprising:
calculating, by the control node based on a maximum flow algorithm, the first value, the fourth value, and the sixth value, the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising:
receiving, by the control node from the target storage node, the quantity of remaining I/O requests of the target storage node in the unit time and the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time.

### Claim 8

A control node, comprising:
an interface; and
a processor coupled to the interface and configured to:
obtain, based on a quantity of remaining input/output (I/O) requests of a target storage node in a unit time, a quantity of I/O requests required by a storage resource to reach a lower assurance limit in the unit time, and a quantity of I/O requests that need to be processed by the target storage node in the unit time, a lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, wherein the target storage node is one of N storage nodes that provide the storage resource in a storage system, N is a positive integer;
allocate a lower limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time to the storage resource based on the lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time; and
send the lower limit quantity of tokens to the target storage node.

### Claim 9 (depends on 8)

The control node according to claim 8, wherein the processor is further configured to:
obtain, based on the quantity of remaining I/O requests of the target storage node in the unit time, a quantity of I/O requests required by the storage resource to reach an upper flow control limit in the unit time, the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time, and the lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, an upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time;
allocate an upper limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time to the storage resource based on the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time; and
sequentially send the upper limit quantity of tokens to the target storage node.

### Claim 10 (depends on 8)

The control node according to claim 8, wherein the processor is further configured to:
calculate, based on a maximum flow algorithm, the quantity of remaining input/output I/O requests of the target storage node in the unit time, the quantity of I/O requests required by the storage resource to reach the lower assurance limit in the unit time, and the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time, the lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time.

### Claim 11 (depends on 9)

The control node according to claim 9, wherein the processor is further configured to:
calculate a first value, a fourth value, and a sixth value, and obtain, based on the first value, the fourth value, and the sixth value, the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, wherein the first value is a difference between a second value and a third value, the second value is the quantity of I/O requests required by the storage resource to reach the upper flow control limit in the unit time, and the third value is a sum of lower limit quantities of I/O requests that can be processed by the N storage nodes for the storage resource in the unit time; the fourth value is a difference between a fifth value and the third value, and the fifth value is the quantity of remaining I/O requests of the target storage node in the unit time; and the sixth value is a difference between a seventh value and the third value, and the seventh value is the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time.

### Claim 12 (depends on 11)

The control node according to claim 11, wherein the processor is further configured to:
allocate an overcommitment value that is of the storage resource on the target storage node and that is in the unit time to the storage resource according to the following formula:
the overcommitment value that is of the storage resource on the target storage node and that is in the
unit


time
=
an


eigth


value
+
(
the


second


value
-
a


ninth


value
)
×
the


seventh


value
a


tenth


value
,
wherein the eighth value is the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, the ninth value is a sum of upper limit quantities of I/O requests that can be processed by the N storage nodes for the storage resource in the unit time, and the tenth value is a sum of quantities of I/O requests that need to be processed by the N storage nodes for the storage resource in the unit time; and
allocate the upper limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time to the storage resource, wherein the upper limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time is a sum of the overcommitment value that is of the storage resource on the target storage node and that is in the unit time and the lower limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time.

### Claim 13 (depends on 11)

The control node according to claim 11, wherein the processor is further configured to:
calculate, based on a maximum flow algorithm, the first value, the fourth value, and the sixth value, the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time.

### Claim 14 (depends on 8)

The control node according to claim 8, wherein the processor is further configured to receive, from the target storage node, the quantity of remaining I/O requests of the target storage node in the unit time and the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time.

### Claim 15

A storage system, comprising a control node and N storage nodes, wherein the control node is configured to:
obtain, based on a quantity of remaining input/output (I/O) requests of a target storage node in a unit time, a quantity of I/O requests required by a storage resource to reach a lower assurance limit in the unit time, and a quantity of I/O requests that need to be processed by the target storage node in the unit time, a lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, wherein the target storage node is one of N storage nodes that provide the storage resource in a storage system, N is a positive integer;
allocate a lower limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time to the storage resource based on the lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time; and
send the lower limit quantity of tokens to the target storage node.

### Claim 16 (depends on 15)

The storage system according to claim 15, wherein the control node is further configured to:
obtain, based on the quantity of remaining I/O requests of the target storage node in the unit time, a quantity of I/O requests required by the storage resource to reach an upper flow control limit in the unit time, the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time, and the lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, an upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time;
allocate an upper limit quantity of tokens that is of the storage resource on the target storage node and that is in the unit time to the storage resource based on the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time; and
sequentially send the upper limit quantity of tokens to the target storage node.

### Claim 17 (depends on 15)

The storage system according to claim 15, wherein the control node is further configured to:
calculate, based on a maximum flow algorithm, the quantity of remaining input/output I/O requests of the target storage node in the unit time, the quantity of I/O requests required by the storage resource to reach the lower assurance limit in the unit time, and the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time, the lower limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time.

### Claim 18 (depends on 16)

The storage system according to claim 16, wherein the control node is further configured to:
calculate a first value, a fourth value, and a sixth value, and obtain, based on the first value, the fourth value, and the sixth value, the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time, wherein the first value is a difference between a second value and a third value, the second value is the quantity of I/O requests required by the storage resource to reach the upper flow control limit in the unit time, and the third value is a sum of lower limit quantities of I/O requests that can be processed by the N storage nodes for the storage resource in the unit time; the fourth value is a difference between a fifth value and the third value, and the fifth value is the quantity of remaining I/O requests of the target storage node in the unit time; and the sixth value is a difference between a seventh value and the third value, and the seventh value is the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time.

### Claim 19 (depends on 18)

The storage system according to claim 18, wherein the control node is further configured to:
calculate, based on a maximum flow algorithm, the first value, the fourth value, and the sixth value, the upper limit quantity of I/O requests that can be processed by the target storage node for the storage resource in the unit time.

### Claim 20 (depends on 15)

The storage system according to claim 15, wherein the control node is further configured to:
receive, from the target storage node, the quantity of remaining I/O requests of the target storage node in the unit time and the quantity of I/O requests that need to be processed by the target storage node for the storage resource in the unit time.
