Erasure-coding-based efficient data storage and retrieval

A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for efficiently storing and retrieving data in a distributed storage system, the method comprising:

• determining, by the computer, a finite field solution that conforms to both locality and maximum distance separable (MDS) properties of an erasure-coding system;
• determining a generator matrix of the erasure-coding system based on the finite field solution;
• generating, from a data element, a plurality of coded fragments based on the generator matrix of the erasure-coding system, wherein the plurality of coded fragments includes a set of enhanced coded fragments that allows reconstruction of the data element and a set of regular coded fragments that facilitates the reconstruction of the data element in conjunction with the set of enhanced coded fragments, and wherein the number of the enhanced coded fragments is fewer than a threshold number of regular coded fragments needed to retrieve the data element using the erasure-coding system;
• and
• providing the plurality of coded fragments for storing in storage devices of the distributed storage system.

A computing system for efficiently storing and retrieving data in a distributed storage system, the computing system comprising:

• a processor;
• and
• a memory coupled to the processor and storing instructions, which when executed by the processor cause the processor to perform a method, the method comprising:
• determining a finite field solution that conforms to both locality and maximum distance separable (MDS) properties of an erasure-coding system;
• determining a generator matrix of the erasure-coding system based on the finite field solution;
• generating, from a data element, a plurality of coded fragments based on the generator matrix of the erasure-coding system, wherein the plurality of coded fragments includes a set of enhanced coded fragments that allows reconstruction of the data element and a set of regular coded fragments that facilitates the reconstruction of the data element in conjunction with the set of enhanced coded fragments, and wherein the number of the enhanced coded fragments is lower than a threshold number of regular coded fragments needed to retrieve the data element using the erasure-coding system;
• and
• providing the plurality of coded fragments for storing in storage devices of the distributed storage system.