# US 2021/0334030 A1 · Storage device parameter monitoring for load balancing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334030 A1
- Document ID: US20210334030A1
- Status: Published application
- Kind: A1
- Application number: 16/855,549
- Filed: 2020-04-22
- Published: 2021-10-28
- Earliest claimed: 2020-04-22
- Assignees: Western Digital Technologies, Inc.
- Applicants: Western Digital Technologies, Inc.
- Inventors: Niles Yang; Phil Reusswig; Sahil Sharma; Rohit Sehgal
- Web page: https://patentgrep.com/patent/US20210334030A1

## CPC

- G06F 3/0635: {by changing the path, e.g. traffic rerouting, path reconfiguration}
- G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors}
- G06F 3/064: {Management of blocks}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G11C 16/349: {Arrangements for evaluating degradation, retention or wearout, e.g. by counting erase cycles}
- G06F 2206/1012: Load balancing
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G11C: STATIC STORES (semiconductor memory devices H10B)

## Abstract

Systems and methods for storage systems using storage device monitoring for load balancing are described. Storage devices may be configured for data access through a common data stream, such as the storage devices in a storage node or server. Data operations from the common data stream may be distributed among the storage devices using a load balancing algorithm. Performance parameter values, such as grown bad blocks, program-erase cycles, and temperature, may be received for the storage devices and used to determine variance values for each storage device. Variance values demonstrating degrading storage devices may be used to reduce the load allocation of data operations to the degrading storage devices.

## Claims

### Claim 1

A storage system, comprising:
a storage system manager configured to:
manage a plurality of storage devices configured for data access through a common data stream; and
distribute data operations from the common data stream to the plurality of storage devices;
a storage device monitor configured to:
receive, for each storage device of the plurality of storage devices, a grown bad block value;
determine, for each storage device of the plurality of storage devices, a first variance value based on grown bad block values from the plurality of storage devices;
determine, based on the first variance value, a degrading storage device that has increased grown bad blocks relative to the plurality of storage devices;
receive an overprovisioning value for the degrading storage device; and
determine, based on the overprovisioning value, that the degrading storage device has remaining overprovisioning space below an overprovisioning response threshold; and
a load balancer configured to:
determine, using a load balancing algorithm, target storage devices among the plurality of storage devices to receive data operations from the common data stream;
reduce, responsive to the determination of the degrading storage device, a load allocation of data operations distributed to the degrading storage device;
allocate, responsive to the remaining overprovisioning space being below the overprovisioning response threshold, the degrading storage device to a secondary storage status; and
skip the degrading storage device for distribution of write data operations from the common data stream.

### Claim 2

(canceled)

### Claim 3 (depends on 1)

The storage system of claim 1, wherein:
the load balancing algorithm is configured to distribute data operations evenly across the plurality of storage devices by selecting target storage devices for data operations based on load allocation values for each storage device of the plurality of storage devices; and
reducing the load allocation of data operations distributed to the degrading storage device is based on changing a load allocation value for the degrading storage device in the load balancing algorithm.

### Claim 4 (depends on 1)

The storage system of claim 1, wherein the storage device monitor is further configured to:
receive, for each storage device of the plurality of storage devices, a program erase cycle value;
determine, for each storage device of the plurality of storage devices, a second variance value based on program erase cycle values from the plurality of storage devices; and
determine, based on the second variance value, that the degrading storage device has increased program erase cycles relative to the plurality of storage devices.

### Claim 5 (depends on 4)

The storage system of claim 4, wherein the storage system manager is further configured to initiate, responsive to the degrading storage device having increased program erase cycles, a reconditioning process to rewrite data blocks stored on the degrading storage device to the degrading storage device.

### Claim 6 (depends on 1)

The storage system of claim 1, wherein the storage device monitor is further configured to:
receive, for each storage device of the plurality of storage devices, a temperature value;
determine, for each storage device of the plurality of storage devices, a third variance value based on temperature values from the plurality of storage devices; and
determine, based on the third variance value, that the degrading storage device has increased temperature relative to the plurality of storage devices.

### Claim 7 (depends on 6)

The storage system of claim 6, wherein the storage system manager is further configured to initiate, responsive to the degrading storage device having increased temperature, an environmental change notification to a cooling system associated with the plurality of storage devices.

### Claim 8 (depends on 1)

The storage system of claim 1, wherein the load balancer is further configured to:
identify a set of data blocks having associated data operations above a hot data threshold; and
allocate the set of data blocks across each of the plurality of storage devices, excluding the degrading storage device.

### Claim 9 (depends on 8)

The storage system of claim 8, wherein:
the load balancer is further configured to rotate the set of data blocks from an initial storage device to the plurality of storage devices, excluding the degrading storage device; and
the load balancer selected the initial storage device to receive the associated data operations for the set of data blocks.

### Claim 10 (depends on 1)

The storage system of claim 1, further comprising:
a storage device performance parameter table configured to store, for each storage device of the plurality of storage devices, performance parameters received from the plurality of storage devices, wherein:
the storage device monitor is further configured to determine the first variance value based on:
aggregating each grown bad block value in successive table entries for each storage device of the plurality of storage devices; and
comparing each grown bad block value to an aggregate value to determine the first variance value for each storage device of the plurality of storage devices; and
the performance parameters include at least one of:
grown bad block values;
program erase cycle values;
temperature values; and
data operations values.

### Claim 11

A computer-implemented method, comprising:
managing a plurality of storage devices configured for data access through a common data stream;
distributing data operations from the common data stream to the plurality of storage devices;
determining, using a load balancing algorithm, target storage devices among the plurality of storage devices to receive data operations from the common data stream;
receiving, for each storage device of the plurality of storage devices, a grown bad block value;
receiving, for each storage device of the plurality of storage devices, a temperature value;
determining, for each storage device of the plurality of storage devices, a first variance value based on grown bad block values from the plurality of storage devices;
determining, for each storage device of the plurality of storage devices, a third variance value based on temperature values from the plurality of storage devices;
determining, based on the first variance value, a degrading storage device that has increased grown bad blocks relative to the plurality of storage devices;
determining, based on the third variance value, that the degrading storage device has increased temperature relative to the plurality of storage devices; and
reducing, responsive to determining that the degrading storage device has increased temperature relative to the plurality of storage devices, a load allocation of data operations distributed to the degrading storage device.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, further comprising:
receiving an overprovisioning value for the degrading storage device;
determining, based on the overprovisioning value, that the degrading storage device has remaining overprovisioning space below an overprovisioning response threshold;
allocating, responsive to the remaining overprovisioning space being below the overprovisioning response threshold, the degrading storage device to a secondary storage status; and
skipping, based on the secondary storage status, the degrading storage device for distribution of write data operations from the common data stream.

### Claim 13 (depends on 11)

The computer-implemented method of claim 11, wherein:
determining target storage devices includes distributing data operations evenly across the plurality of storage devices by selecting target storage devices for data operations based on load allocation values for each storage device of the plurality of storage devices; and
reducing the load allocation of data operations distributed to the degrading storage device is based on changing a load allocation value for the degrading storage device in the load balancing algorithm.

### Claim 14 (depends on 11)

The computer-implemented method of claim 11, further comprising:
receiving, for each storage device of the plurality of storage devices, a program erase cycle value;
determining, for each storage device of the plurality of storage devices, a second variance value based on program erase cycle values from the plurality of storage devices; and
determining, based on the second variance value, that the degrading storage device has increased program erase cycles relative to the plurality of storage devices.

### Claim 15 (depends on 14)

The computer-implemented method of claim 14, further comprising:
initiating, responsive to the degrading storage device having increased program erase cycles, a reconditioning process to rewrite data blocks stored on the degrading storage device to the degrading storage device.

### Claim 16

(canceled)

### Claim 17 (depends on 11)

The computer-implemented method of claim 11, further comprising:
initiating, responsive to the degrading storage device having increased temperature, an environmental change notification to a cooling system associated with the plurality of storage devices.

### Claim 18 (depends on 11)

The computer-implemented method of claim 11, further comprising:
identifying a set of data blocks having associated data operations above a hot data threshold; and
allocating the set of data blocks across each of the plurality of storage devices, excluding the degrading storage device.

### Claim 19 (depends on 18)

The computer-implemented method of claim 18, further comprising:
rotating, from an initial storage device among the plurality of storage devices, the set of data blocks to the plurality of storage devices, excluding the degrading storage device, wherein the initial storage device received the associated data operations for the set of data blocks.

### Claim 20

A system, comprising:
a plurality of storage devices configured for data access through a common data stream;
means for distributing data operations from the common data stream to the plurality of storage devices;
means for determining, using a load balancing algorithm, target storage devices among the plurality of storage devices to receive data operations from the common data stream;
means for receiving, for each storage device of the plurality of storage devices, a grown bad block value;
means for receiving, for each storage device of the plurality of storage devices, a program erase cycle value;
means for determining, for each storage device of the plurality of storage devices, a first variance value based on grown bad block values from the plurality of storage devices;
means for determining, for each storage device of the plurality of storage devices, a second variance value based on program erase cycle values from the plurality of storage devices;
means for determining, based on the first variance value, a degrading storage device that has increased grown bad blocks relative to the plurality of storage devices;
means for determining, based on the second variance value, that the degrading storage device has increased program erase cycles relative to the plurality of storage devices; and
means for reducing, responsive to determining that the degrading storage device has increased program erase cycles relative to the plurality of storage devices, a load allocation of data operations distributed to the degrading storage device.

### Claim 21 (depends on 20)

The system of claim 20, further comprising:
means for initiating, responsive to the degrading storage device having increased program erase cycles, a reconditioning process to rewrite data blocks stored on the degrading storage device to the degrading storage device.

### Claim 22 (depends on 20)

The system of claim 20, further comprising:
means for receiving an overprovisioning value for the degrading storage device;
means for determining, based on the overprovisioning value, that the degrading storage device has remaining overprovisioning space below an overprovisioning response threshold;
means for allocating, responsive to the remaining overprovisioning space being below the overprovisioning response threshold, the degrading storage device to a secondary storage status; and
means for skipping, based on the secondary storage status, the degrading storage device for distribution of write data operations from the common data stream.
