# US 2025/0094295 A1 · Use of cluster-level redundancy within a cluster of a distributed storage management system to address node-level errors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094295 A1
- Document ID: US20250094295A1
- Status: Published application
- Kind: A1
- Application number: 18/962,013
- Filed: 2024-11-27
- Published: 2025-03-20
- Earliest claimed: 2021-11-16
- Assignees: NetApp, Inc.
- Applicants: NetApp, Inc.
- Inventors: Wei Sun; Anil Paul Thoppil; Anne Maria Vasu
- Web page: https://patentgrep.com/patent/US20250094295A1

## CPC

- G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors}
- G06F 11/1662: {the resynchronized component or unit being a persistent storage device (re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008)}
- G06F 3/064: {Management of blocks}
- G06F 3/0689: {Disk arrays, e.g. RAID, JBOD}
- G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems}
- G06F 11/1088: {Reconstruction on already foreseen single or plurality of spare disks}
- G06F 11/1092: {Rebuilding, e.g. when physically replacing a failing disk}
- G06F 11/2094: {Redundant storage or storage space (G06F11/2056 takes precedence)}
- G06F 11/3034: {where the computing system component is a storage system, e.g. DASD based or network based (digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods that make use of cluster-level redundancy within a distributed storage management system to address various node-level error scenarios are provided. According to one embodiment, an instance of a key-value (KV) store of a first node of a plurality of nodes of a cluster of a distributed storage system manages storage of data blocks as values and corresponding block identifiers (IDs) as keys. A list of missing block IDs that are in use for one or more volumes associated with the first node but that are missing from the instance of the KV store are identified by performing a data integrity check on the instance of the KV store. After identifying the list of missing block IDs, instead of treating the first node as failed, restoring the missing block IDs by writing redundant data blocks retrieved from other nodes within the cluster to the first node.

## Claims

### Claim 1

A method comprising:
managing, by an instance of a key-value (KV) store of a first node of a plurality of nodes of a cluster of a distributed storage system, storage of data blocks as values and corresponding block identifiers (IDs) as keys;
identifying a list of missing block IDs that are in use for one or more volumes associated with the first node but that are missing from the instance of the KV store by performing a data integrity check on the instance of the KV store; and
after identifying the list of missing block IDs, avoiding treating the first node as failed and instead performing a resynchronization process by, for each block ID in the list of block IDs:
reading a data block corresponding to the block ID from a second node of the plurality of nodes that maintains a redundant data block corresponding to the block ID; and
restoring the block ID within the KV store by writing the redundant data block to the first node.

### Claim 2 (depends on 1)

The method of claim 1, wherein the second node is one of multiple nodes of the plurality of nodes that maintains the redundant data block.

### Claim 3 (depends on 1)

The method of claim 1, wherein the data integrity check is separate from and performed independently of a garbage collection process.

### Claim 4 (depends on 1)

The method of claim 1, wherein the data integrity check is integrated with and performed as part of a garbage collection process.

### Claim 5 (depends on 1)

The method of claim 1, wherein the block IDs in the list of missing block IDs are associated with one or more bins of a plurality of bins each representing a subset of block IDs in a range of block IDs, wherein each of the one or more bins are assigned to one or more block services running on the first node and a respective redundant block service within the cluster, and wherein the one or more block services and the respective redundant block service are operable to store and retrieve data blocks associated with the assigned bins based on their respective block IDs.

### Claim 6 (depends on 5)

The method of claim 5, further comprising identifying the redundant block service based on a bin of the one or more bins with which the block ID is associated, wherein an association between a given block ID and a given bin is determined based on a subset of bits of the given block ID.

### Claim 7

A non-transitory machine readable medium storing instructions, which when executed by one or more processing resources of a cluster of a plurality of nodes representing a distributed storage system, cause the distributed storage system to:
identify a list of missing block identifiers (IDs) that are in use for one or more volumes associated with a first node of the plurality of nodes but that are missing from an instance of a key-value (KV) store of the first node by performing a data integrity check on the instance of the KV store; and
after identifying the list of missing block IDs, avoiding treating the first node as failed and instead performing a resynchronization process by, for each block ID in the list of block IDs:
reading a data block corresponding to the block ID from a second node of the plurality of nodes that maintains a redundant data block corresponding to the block ID; and
restoring the block ID within the KV store by writing the redundant data block to the first node.

### Claim 8 (depends on 7)

The non-transitory machine readable medium of claim 7, wherein the data integrity check involves performing a walkthrough of mapped logical block addresses to identify block IDs that are currently in use for the one or more volumes.

### Claim 9 (depends on 7)

The non-transitory machine readable medium of claim 7, wherein the second node is one of multiple nodes of the plurality of nodes that maintains the redundant data block.

### Claim 10 (depends on 7)

The non-transitory machine readable medium of claim 7, wherein the data integrity check is separate from and performed independently of a garbage collection process.

### Claim 11 (depends on 7)

The non-transitory machine readable medium of claim 7, wherein the data integrity check is integrated with and performed as part of a garbage collection process.

### Claim 12 (depends on 7)

The non-transitory machine readable medium of claim 7, wherein the block IDs in the list of missing block IDs are associated with one or more bins of a plurality of bins each representing a subset of block IDs in a range of block IDs, wherein each of the one or more bins are assigned to one or more block services running on the first node and a respective redundant block service within the cluster, and wherein the one or more block services and the respective redundant block service are operable to store and retrieve data blocks associated with the assigned bins based on their respective block IDs.

### Claim 13 (depends on 12)

The non-transitory machine readable medium of claim 12, wherein the instructions further cause the distributed storage system to identify the redundant block service based on a bin of the one or more bins with which the block ID is associated, wherein an association between a given block ID and a given bin is determined based on a subset of bits of the given block ID.

### Claim 14

A distributed storage system comprising:
a plurality of nodes operating as a cluster and including one or more processing resources and instructions that when executed by the one or more processing resources cause the distributed storage system to:
manage, within an instance of a key-value (KV) store of a first node of the plurality of nodes, storage of data blocks as values and corresponding block identifiers (IDs) as keys;
avoiding treating the first node as failed and instead performing a resynchronization process by, for each block ID in a list of missing block IDs that are in use for one or more volumes associated with the first node but that are missing from the instance of the KV store:
reading a data block corresponding to the block ID from a second node of the plurality of nodes that maintains a redundant data block corresponding to the block ID; and
restoring the block ID within the KV store by writing the redundant data block to the first node.

### Claim 15 (depends on 14)

The distributed storage system of claim 14, wherein the data integrity check involves performing a walkthrough of mapped logical block addresses to identify block IDs that are currently in use for the one or more volumes.

### Claim 16 (depends on 14)

The distributed storage system of claim 14, wherein the second node is one of multiple nodes of the plurality of nodes that maintains the redundant data block.

### Claim 17 (depends on 14)

The distributed storage system of claim 14, wherein the list of missing block IDs are identified by performing a data integrity check on the instance of the KV store.

### Claim 18 (depends on 17)

The distributed storage system of claim 17, wherein the data integrity check is separate from and performed independently of a garbage collection process.

### Claim 19 (depends on 17)

The distributed storage system of claim 17, wherein the data integrity check is integrated with and performed as part of a garbage collection process.

### Claim 20 (depends on 14)

The distributed storage system of claim 14, wherein the block IDs in the list of missing block IDs are associated with one or more bins of a plurality of bins each representing a subset of block IDs in a range of block IDs, wherein each of the one or more bins are assigned to one or more block services running on the first node and a respective redundant block service within the cluster, and wherein the one or more block services and the respective redundant block service are operable to store and retrieve data blocks associated with the assigned bins based on their respective block IDs.

### Claim 21 (depends on 19)

The distributed storage system of claim 19, wherein the instructions further cause the distributed storage system to identify the redundant block service based on a bin of the one or more bins with which the block ID is associated, wherein an association between a given block ID and a given bin is determined based on a subset of bits of the given block ID.
