Fault tolerant distributed storage method and controller using (N,K) algorithms

A method for data storage in a distributed data storage system with redundancy, the method comprising:

• dividing a data set into a plurality of data blocks;
• defining a size of data block and adding filler to the data set so that the data set is represented by an integral number of data blocks
• defining a size of data block and adding filler to any data block that is less than the defined size;
• defining a minimal number K out of N data chunks needed to restore one data block;
• disassembling each of the data blocks into at least L different data chunks, wherein K≦L≦N;
• and
• distributing the at least L data chunks to storage elements of the distributed storage system.

A method for retrieving data in a distributed data storage system comprising:

• receiving, from a distributed storage system, any K data chunks out of L data chunks for each data block, wherein K≦L;
• composing the received data chunks into corresponding data blocks;
• assembling the data blocks into a data set;
• and
• removing added data from the data set, the added data resulting from chunk generation.

A system for managing distributed storage comprising:

• decomposition logic that disassembles each data block into L different data chunks, such that K data chunks out of N data chunks are sufficient to restore the original data blocks, wherein L≦N;
• an interface for distributing data to storage elements;
• composition logic for assembling K data chunks received for each data block into corresponding data blocks;
• and
• control logic to control operations of the decomposition logic and the composition logic.

A system for managing distributed storage comprising:

• decomposition logic that disassembles each data block into L different data chunks, such that K data chunks out of N data chunks are sufficient to restore the original data blocks, wherein L≦N;
• an interface for distributing data to storage elements;
• composition logic for assembling K data chunks received for each data block into corresponding data blocks;
• and
• control logic to control operations of the decomposition logic and the composition logic,
• wherein any of N, L, and K vary for different data blocks.