Hybrid distributed storage system to dynamically modify storage overhead and improve access performance

An apparatus comprising:

• a network interface that enables network communications;
• a processor;
• and
• a memory to store data and instructions executable by the processor,
• wherein the processor is configured to execute the instructions to:
• store an object according to a first storage policy in a distributed storage system that includes a plurality of storage nodes, the storage policy specifying one or a combination of replication and erasure coding to redundantly store the object, wherein storing the object according to the first storage policy results in a first storage overhead for the object;
• determine a popularity of the object based on a number of client requests for retrieval of the object, the popularity of the object being specified by a popularity score that is updated in response to receipt of the client requests for retrieval of the object;
• in response to a change in the popularity of the object indicated by the popularity score crossing a popularity threshold, change from the first storage policy to a second storage policy for the object;
• and
• store the object according to the second storage policy that results in a second storage overhead for the object different from the first storage overhead due to the second storage policy specifying a different combination of replication and erasure coding from the first storage policy or by specifying one of replication and erasure coding not specified by the first storage policy, wherein the object is stored by erasure coding in response to the popularity score being below the popularity threshold, and the object is stored by replication in response to the popularity score being above the popularity threshold.

A non-transitory computer-readable storage medium encoded with software comprising computer executable instructions which, when executed by a processor, cause the processor to:

• store an object according to a first storage policy in a distributed storage system that includes a plurality of storage nodes, the storage policy specifying one or a combination of replication and erasure coding to redundantly store the object, wherein storing the object according to the first storage policy results in a first storage overhead for the object;
• determine a popularity of the object based on a number of client requests for retrieval of the object, the popularity of the object being specified by a popularity score that is updated in response to receipt of the client requests for retrieval of the object;
• in response to a change in the popularity of the object indicated by the popularity score crossing a popularity threshold, change from the first storage policy to a second storage policy for the object;
• and
• store the object according to the second storage policy that results in a second storage overhead for the object different from the first storage overhead due to the second storage policy specifying a different combination of replication and erasure coding from the first storage policy or by specifying one of replication and erasure coding not specified by the first storage policy wherein the object is stored by erasure coding in response to the popularity score being below the popularity threshold, and the object is stored by replication in response to the popularity score being above the popularity threshold.