Processing and distributing write logs of nodes of a cluster storage system

A method for implementing a cluster storage system for providing clients access to data of a shared storage, the method comprising:

• storing data to the shared storage, the shared storage comprising a plurality of aggregates, each aggregate comprising a predetermined set of storage devices for storing data of the aggregate;
• providing a plurality of nodes, each node assigned to service data of a predetermined plurality of aggregates in the shared storage, wherein during normal operation when no node failure has occurred, each node can access only the aggregates assigned to the node, the plurality of nodes comprising a primary node and a plurality of partner nodes comprising a first partner node and a second partner node;
• assigning the primary node to service data of a first aggregate and a second aggregate;
• at the primary node:
• producing a plurality of write logs for a plurality of received write requests for the first and second aggregates;
• producing parity data for the plurality of write logs;
• storing the parity data to local non-volatile storage, wherein the plurality of write logs are not stored to local non-volatile storage;
• dividing the plurality of write logs on a per aggregate basis, wherein the plurality of write logs is divided into a first set of write logs for the first aggregate and a second set of write logs for the second aggregate;
• and
• distributing the first set of write logs only to the first partner node and the second set of write logs only to the second partner node;
• at the first partner node:
• receiving the first set of write logs for the first aggregate;
• and
• performing the first set of write logs on the first aggregate upon failure of the primary node, wherein the first partner node is the only partner node performing write logs on the first aggregate upon failure of the primary node;
• and
• at the second partner node:
• receiving the second set of write logs for the second aggregate;
• and
• performing the second set of write logs on the second aggregate upon failure of the primary node, wherein the second partner node is the only partner node performing write logs on the second aggregate upon failure of the primary node.

A non-transitory computer readable medium having instructions stored thereon when executed, implement a cluster storage system for providing clients access to data of a shared storage, the computer readable medium comprising sets of instructions for:

• storing data to the shared storage, the shared storage comprising a plurality of aggregates, each aggregate comprising a predetermined set of storage devices for storing data of the aggregate;
• providing a plurality of nodes, each node assigned to service data of a predetermined plurality of aggregates in the shared storage, wherein during normal operation when no node failure has occurred, each node can access only the aggregates assigned to the node, the plurality of nodes comprising a primary node and a plurality of partner nodes comprising a first partner node and a second partner node;
• assigning the primary node to service data of a first aggregate and a second aggregate;
• at the primary node:
• producing a plurality of write logs for a plurality of received write requests for the first and second aggregates;
• producing parity data for the plurality of write logs;
• storing the parity data to local non-volatile storage, wherein the plurality of write logs are not stored to local non-volatile storage;
• dividing the plurality of write logs on a per aggregate basis, wherein the plurality of write logs is divided into a first set of write logs for the first aggregate and a second set of write logs for the second aggregate;
• and
• distributing the first set of write logs only to the first partner node and the second set of write logs only to the second partner node;
• at the first partner node:
• receiving the first set of write logs for the first aggregate;
• and
• performing the first set of write logs on the first aggregate upon failure of the primary node, wherein the first partner node is the only partner node performing write logs on the first aggregate upon failure of the primary node;
• and
• at the second partner node:
• receiving the second set of write logs for the second aggregate;
• and
• performing the second set of write logs on the second aggregate upon failure of the primary node, wherein the second partner node is the only partner node performing write logs on the second aggregate upon failure of the primary node.