Dynamic distribution of replicated data

A distributed storage system, comprising:

• a computer processor;
• a cluster map representing a plurality of storage resources;
• a distribution engine executing on the computer processor and configured to:
• identifying a first virtual bucket of a plurality of virtual buckets;
• for each virtual bucket of the plurality of virtual buckets:
• using an identifier of the virtual bucket and the cluster map to identify a primary storage resource;
• identifying a candidate resource set mapped to by the primary storage resource;
• using an identifier of the virtual bucket to traverse a subset of the cluster map corresponding to storage resources in the candidate resource set to identify one or more secondary storage resources;
• and
• mapping the virtual bucket to a final resource set identifying the primary storage resource and the secondary storage resources;
• a frontend module configured to:
• receive a request to store a data object;
• selecting a particular virtual bucket of the plurality of virtual buckets based on an identifier of the data object;
• identifying a corresponding final resource set mapped to by the selected virtual bucket;
• and
• storing the data object in each storage resource of the final resource set.

A non-transitory computer readable medium comprising computer readable program code for:

• identifying a first virtual bucket of a plurality of virtual buckets;
• for each virtual bucket of the plurality of virtual buckets:
• using an identifier of the virtual bucket and a cluster map representing a plurality of storage resources to identify a primary storage resource;
• identifying a candidate resource set mapped to by the primary storage resource;
• using an identifier of the virtual bucket to traverse a subset of the cluster map corresponding to storage resources in the candidate resource set to identify one or more secondary storage resources;
• and
• mapping the virtual bucket to a final resource set identifying the primary storage resource and the secondary storage resources;
• identifying a data object associated with a storage request;
• selecting a particular virtual bucket of the plurality of virtual buckets based on an identifier of the data object;
• identifying a corresponding final resource set mapped to by the selected virtual bucket;
• and
• storing the data object in each storage resource of the final resource set.