US 10,503,611 B1 · Data protection management for distributed storage

Abstract

An efficient and scalable method of managing data protection in a geographically distributed storage system is provided using erasure coding. Each zone in the geographically distributed storage system of multiple zones or clusters encodes primary backup chunks replicated from original data chunks in remote zones to create coding chunks. The zone retains one of the coding chunks and distributes the remaining coding chunks to remote zones for future recovery from dual and more complex failures. Upon completion, the primary backup chunks and peer backup chunks can safely be deleted to reduce overhead. Encoding k primary backup chunks to create m coding chunks, and distributing the m coding chunks to different remote zones assures recovery from 1 to m zone/cluster/chunk failures. After 1 to m zone failures the unaffected zones are each able to recover all of the unavailable chunks by retrieving from each other any of the unaffected zone's distributed coding chunks and still available original data chunks, and restoring the content of the unavailable chunks by decoding the combination of the retained coding chunk, distributed coding chunk, and any still available original data chunks.

Claims19

Dependent claims

A computer-executable method of managing data protection in a distributed storage system, the method comprising:

configuring a distributed storage system into a plurality of at least k+m zones in which each zone accumulates k equal-sized primary backup chunks and one or more peer backup chunks of original data chunks replicated from different remote zones;
and
preparing the distributed storage system for recovery from a failure of 1 to m zones of the plurality of the at least k+m zones by performing the operations of:
encoding, at each zone of the plurality of at least k+m zones, the k equal-sized primary backup chunks to create m equalized-sized redundant coding chunks using an erasure coding such that the distributed storage system has at least k+m chunks of data,
retaining one of the m coding chunks in that zone of the plurality of at least k+m zones,
distributing each of the remaining m−1 coding chunks to a different one of the remaining m−1 zones, and
deleting each of k equal-sized primary backup chunks and the one or more peer backup chunks in that zone of plurality of at least k+m zones to reduce storage overhead.

4 more elements

A distributed storage system comprising:

a cluster of nodes in communication with one or more data storage arrays and a storage engine, each node having a processor configured to execute an application, the storage engine enabling the application to process data I/O requests to and from the one or more data storage arrays, the processor further configured to:
configure the cluster of nodes into a plurality of k+m zones in which each zone manages data stored in the one or more data storage arrays, the data including k equal-sized primary backup chunks and one or more peer backup chunks of original data chunks replicated from different remote zones;
and
prepare the cluster of nodes for recovery from a failure of any one or more zones in the plurality of zones, including causing each zone in the plurality of zones to:
encode the equal-sized primary backup chunks to create m coding chunks using erasure coding such that the distributed storage system has at least k+m chunks of data,
retain one of the m coding chunks in the zone,
distribute each of the other m−1 coding chunks not retained to different zones in the plurality of zones, and
delete any primary backup chunks and any peer backup chunks from the zone to reduce storage overhead.

5 more elements

A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for data protection management in a distributed storage system, the operations comprising:

configuring a cluster of nodes into a plurality of k+m zones in which each zone manages data stored in the one or more data storage arrays, the data including k equal-sized primary backup chunks and one or more peer backup chunks of original data chunks replicated from different remote zones;
and
preparing the cluster of nodes for recovery from a failure of any one or more zones in the plurality of zones, including causing each zone in the plurality of zones to:
encode the k equal-sized primary backup chunks to create in coding chunks using erasure coding,
retain one of the m coding chunks in the zone,
distribute each of the other m−1 coding chunks not retained to different zones in the plurality of zones, and
delete any primary backup chunks from which coding chunks were created and any peer backup chunks from the zone to reduce storage overhead.

4 more elements

Dependencies are inferred from the claim text.

Data protection management for distributed storage

Life of this patent

Data protection management for distributed storage

Life of this patent

Abstract

Claims19