Failover mechanism for distributed process execution

A non-transitory computer-readable medium including program code configured to provide operations comprising:

• maintaining a state of a cluster comprising at least a plurality of nodes including a plurality of processors, a controller of the cluster, and a database, wherein a structural change is caused by at least one of adding a first node to the cluster and dropping a second node from the cluster, the structural change causing a change in state, wherein maintaining the state further comprises:
• providing an indication announcing that the structural change to the cluster will occur, wherein the indication is provided by a node initiating the structural change, the indication enabling at least one node of the cluster to respond, after receiving a communication from the node initiating the structural change, with an existing view representing a topology defining the plurality of nodes from the perspective of the at least one node, the topology of defining an ordering of the plurality of nodes according to a sorting criterion, the existing view including a first set of authoritative data representing existing processes executed at the at least one node and processes for which the at least one node owns an authoritative index for but will not be authoritative subsequent to the structural change, wherein each node in the cluster independently determines the processes for which the node will not be authoritative for subsequent to the structural change;
• providing, by the node initiating the structural change, to the plurality of nodes in the cluster an index representing a subsequent view of the cluster after the structural change, the subsequent view including a second set of authoritative data representing processes executed at the at least one node after the structural change, wherein each node in the plurality of nodes receiving the index determines at least one process, for which the node will be authoritative subsequent to the structural change;
• and
• activating, by the node initiating the structural change, based on the index, the subsequent view of the cluster, when the provided index is received at the at least one node;
• wherein, based on the activating, each node in the plurality of nodes, based on the index,
• generates an index of processes that will be executed on the node;
• generates a new cluster topology and provides the new cluster topology to the plurality of nodes in the cluster;
• and
• removes at least one process identified in a previous index and the previous index associated with the node from the generated index of processes.

A system comprising:

• a processor;
• and
• a memory, the processor and memory configured to provide a method comprising:
• maintaining a state of a cluster comprising at least a plurality of nodes including a plurality of processors, a controller of the cluster, and a database, wherein a structural change is caused by at least one of adding a first node to the cluster and dropping a second node from the cluster, the structural change causing a change in state, wherein maintaining the state further comprises:
• providing an indication announcing that the structural change to the cluster will occur, wherein the indication is provided by a node initiating the structural change, the indication enabling at least one node of the cluster to respond, after receiving a communication from the node initiating the structural change, with an existing view representing a topology defining the plurality of nodes from the perspective of the at least one node, the topology of defining an ordering of the plurality of nodes according to a sorting criterion, the existing view including a first set of authoritative data representing existing processes executed at the at least one node and processes for which the at least one node owns an authoritative index for but will not be authoritative subsequent to the structural change, wherein each node in the cluster independently determines the processes for which the node will not be authoritative for subsequent to the structural change;
• providing, by the node initiating the structural change, to the plurality of nodes in the cluster an index representing a subsequent view of the cluster after the structural change, the subsequent view including a second set of authoritative data representing processes executed at the at least one node after the structural change, wherein each node in the plurality of nodes receiving the index determines at least one process, for which the node will be authoritative subsequent to the structural change;
• and
• activating, by the node initiating the structural change, based on the index, the subsequent view of the cluster, when the provided index is received at the at least one node;
• wherein, based on the activating, each node in the plurality of nodes, based on the index,
• generates an index of processes that will be executed on the node;
• generates a new cluster topology and provides the new cluster topology to the plurality of nodes in the cluster;
• and
• removes at least one process identified in a previous index and the previous index associated with the node from the generated index of processes.