Methods for detecting split brain in a distributed system

A method for detecting a split brain condition in a cluster of a plurality of instances comprising:

• detecting, by a second instance, that a first instance of the plurality of instances in the cluster is presumably no longer an active member of the cluster, wherein the first instance comprising a first redo thread generating log records that describe changes to a shared data resource by the first instance;
• updating, by the second instance, a first record in a control file associated with said first redo thread to indicate that the first redo thread is closed;
• and
• periodically checking the control file by the first instance;
• detecting, by the first instance, in the control file, the first record indicating that the first redo thread is closed;
• upon detecting the first record, determining, by the first instance, that the first instance is a member of a disjoint cluster;
• initiating one or more disjoint cluster recovery techniques on the first instance;
• wherein the method is performed by one or more computing devices.

A computer-readable non-transitory storage medium storing one or more instructions for detecting a split brain condition in a cluster of a plurality of instances, which when executed, cause one or more processors to perform:

• determining, by one or more second instances of the plurality of instances, that a first instance of the plurality of instances is presumably no longer an active member of the cluster;
• said first instance, updating a first redo log with log records that describe changes made to a shared data resource by the first instance;
• performing, by the one or more second instances of said plurality of instances, instance recovery of the first instance after determining that the first instance is no longer an active member of the cluster, wherein instance recovery comprises performing a first pass of the redo log and detecting, during a second pass of the redo log, any new activity in the redo log since the first pass;
• and
• based on the new activity, determining that the previously-presumed inactive first instance is an active member of a disjoint cluster.

A computer-readable non-transitory storage medium storing one or more instructions for detecting a split brain condition in a cluster of a plurality of instances, which when executed, cause one or more processors to perform:

• detecting, by a second instance, that a first instance of the plurality of instances is presumably no longer an active member of the cluster,
• wherein the first instance comprises a first redo thread generating log records that describe changes to a shared data resource by the first instance;
• updating, by the second instance, a first record in a control file associated with said first redo thread to indicate that the first redo thread is closed;
• periodically checking the control file by the first instance;
• detecting, by the first instance, in the control file, the first record indicating that the first redo thread is closed;
• upon detecting the first record, determining, by the first instance, that the first instance is a member of a disjoint cluster;
• initiating one or more disjoint cluster recovery techniques on the first instance.