Method and system for a sequence aware data ingest and a sequence aware replication between data clusters

A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for managing data, the method comprising:

• obtaining data from a host;
• performing an erasure coding procedure to the data to obtain a plurality of slices, wherein each slice in the plurality of slices comprises a plurality of data chunks and at least one parity chunk;
• generating a plurality of segment entries, wherein each segment entry in the plurality of segment entries specifies a segment;
• generating a plurality of metadata slice entries, wherein each metadata slice entry in the plurality of metadata slice entries is associated with a slice in the plurality of slices;
• storing the plurality of segment entries and the plurality of metadata slice entries in an accelerator pool in a first data cluster;
• and
• storing, across a plurality of fault domains in the first data cluster, the plurality of data chunks and the at least one parity chunk of each slice in the plurality of slices based on the plurality of segment entries.

A data cluster, comprising:

• an accelerator pool;
• and
• a non-accelerator pool,
• wherein a data processor of the accelerator pool comprises a processor and memory comprising instructions, which when executed by the processor perform a method, the method comprising:
• obtaining data from a host;
• performing an erasure coding procedure to the data to obtain a plurality of slices, wherein each slice in the plurality of slices comprises a plurality of data chunks and at least one parity chunk;
• generating a plurality of segment entries, wherein each segment entry in the plurality of segment entries specifies a segment;
• generating a plurality of metadata slice entries, wherein each metadata slice entry in the plurality of metadata slice entries is associated with a slice in the plurality of slices;
• storing the plurality of segment entries and the plurality of metadata slice entries in an accelerator pool in a first data cluster;
• and
• storing, across a plurality of fault domains in the first data cluster, the plurality of data chunks and the at least one parity chunk of each slice in the plurality of slices based on the plurality of segment entries.