Data tiering in a cluster network using distributed shared logging with multiple segment allocation

A method of storing log data in a tiered storage system for a cluster system using a distributed log, comprising:

• deploying memory resources comprising non-volatile RAM (NVRAM) and tiered storage comprising memory devices of lesser cost and lower performance than the NVRAM devices;
• storing logs of system activity generated by nodes in the cluster system in the NVRAM, wherein the logs are stored in a shared log structure that is segmented into a plurality of segments based on the logs and respective entries;
• maintaining in one node of the nodes, a metastore containing a mapping of the logs with respective storage locations in the respective storage devices;
• and
• migrating logs from the NVRAM to the tiered storage based on at least one of time and space thresholds.

A system migrating log data to tiered storage units in a cluster system, comprising:

• a plurality of nodes, wherein each node generates logs for system activity and maintained in a respective client, an associated client library, a respective log server, and a respective storage device comprising non-volatile RAM (NVRAM) devices;
• tiered storage comprising memory devices of lesser cost and lower performance than the NVRAM devices;
• a shared log structure maintained in storage and storing the logs in a shared log structure that is segmented based on the logs and respective entries;
• a metastore component maintaining in one node of the plurality of nodes, a mapping of the logs with respective storage locations in the respective storage devices;
• and
• a tiering component identifying a segment of the plurality of segments that is sealed against modification or full of data and migrating the segment to the tiered storage based on at least one of time and space threshold values.