Supporting distributed and local objects using a multi-writer log-structured file system

A computer system for supporting distributed and local objects using a multi-writer log-structured file system (LFS), the computer system comprising:

• a processor;
• and
• a non-transitory computer readable medium having stored thereon program code for transferring data to another computer system, the program code causing the processor to:
• on a first node, receive incoming data from each of a plurality of objects local to the first node, wherein the plurality of objects is configured to simultaneously write to the multi-writer LFS;
• coalesce the received incoming data;
• determine whether the coalesced incoming data comprises at least a full segment of data;
• based at least on determining that the coalesced incoming data comprises at least a full segment of data, write at least a first portion of the coalesced incoming data as one or more full segments of data to a first storage of the multi-writer LFS, wherein the coalesced incoming data comprises the first portion and a remainder portion;
• write the remainder portion of the coalesced incoming data to a second storage of the multi-writer LFS;
• acknowledge the writing to the plurality of objects;
• determine whether at least a full segment of data has accumulated in the second storage;
• and
• based at least on determining that at least a full segment of data has accumulated in the second storage, write at least a portion of the accumulated data in the second storage as one or more full segments of data to the first storage.

A non-transitory computer readable storage medium having stored thereon program code executable by a first computer system at a first site, the program code embodying a method comprising:

• on a first node, receiving incoming data from each of a plurality of objects local to the first node, wherein the plurality of objects is configured to simultaneously write to the multi-writer LFS;
• coalescing the received incoming data;
• determining whether the coalesced incoming data comprises at least a full segment of data;
• based at least on determining that the coalesced incoming data comprises at least a full segment of data, writing at least a first portion of the coalesced incoming data as one or more full segments of data to a first storage of the multi-writer LFS, wherein the coalesced incoming data comprises the first portion and a remainder portion;
• writing the remainder portion of the coalesced incoming data to a second storage of the multi-writer LFS;
• acknowledging the writing to the plurality of objects;
• determining whether at least a full segment of data has accumulated in the second storage;
• and
• based at least on determining that at least a full segment of data has accumulated in the second storage, writing at least a portion of the accumulated data in the second storage as one or more full segments of data to the first storage.