Incremental erasure coding for storage systems

A system comprising:

• one or more processors;
• a volatile memory;
• and
• a non-volatile memory storing computer program code that when executed on the processor causes execution across the one or more processors of a process operable to perform the operations of:
• generating a storage chunk having a plurality of data fragments, the storage chunk stored in one or more storage devices;
• allocating space in a primary memory to store a plurality of coded fragments;
• receiving a plurality of I/O requests to write data;
• allocating space in the primary memory to store a next unfilled data fragment;
• processing a plurality of I/O requests to write data, including for each I/O request:
• writing the data to the next unfilled data fragment in the one or more storage devices;
• writing the data to the next unfilled data fragment in the primary memory;
• and
• if the data fragment in the primary memory is full, updating the coded fragments in the primary memory using the filled data fragment in the primary memory;
• and
• copying the coded fragments from the primary memory to the one or more storage devices.

A computer program product tangibly embodied in a non-transitory computer-readable medium, the computer-readable medium storing program instructions that are executable to:

• generate a storage chunk having a plurality of data fragments, the storage chunk stored in one or more storage devices;
• allocate space in a primary memory to store a plurality of coded fragments;
• receive a plurality of I/O requests to write data;
• allocate space in the primary memory to store a next unfilled data fragment;
• process a plurality of I/O requests to write data, including for each I/O request:
• write the data to the next unfilled data fragment in the one or more storage devices;
• write the data to the next unfilled data fragment in the primary memory;
• and
• if the data fragment in the primary memory is full, update the coded fragments in the primary memory using the filled data fragment in the primary memory;
• and
• copy the coded fragments from the primary memory to the one or more storage devices.