Systems and methods for efficiently identifying failed computing nodes

A multi-node computing system, comprising:

• a plurality of computing nodes, each computing node of said plurality of computing nodes having access to associated node memory, for storing data and code, and at least one hardware processor configured to execute said code, said code including a set of native instructions configured to cause said at least one hardware processor to perform a corresponding set of native operations when executed by said at least one hardware processor;
• and
• shared data storage accessible to each node of said plurality of computing nodes, said shared data storage including task information and node information, said task information indicative of a plurality of tasks each available to be processed by one of said nodes, said node information indicative of a status of each of said nodes;
• and wherein
• said data and said code associated with a first one of said nodes include
• a first subset of said set of native instructions configured to generate a first unique identifier particularly corresponding to said associated first node,
• a second subset of said set of native instructions configured to periodically update said node information in said shared data storage with new information associated with said first unique identifier of said first node,
• a third subset of said set of native instructions configured to determine whether a first task, previously identified by said task information as being processed by another node, is available for processing by said first node, and
• a fourth subset of said set of native instructions configured to process said first task when it is determined to be available;
• and
• said task information includes a plurality of records, each record including
• a first field including information identifying a particular task,
• a second field including information identifying a particular status associated with said particular task identified by said information of said first field, and
• a third field including information identifying a particular node of said plurality of computing nodes associated with said particular task identified by said information of said first field.

A method for identifying failed computing nodes, said method comprising:

• providing a plurality of computing nodes, each computing node of said plurality of computing nodes having access to at least one hardware processor and to shared data storage;
• generating a first unique identifier particularly corresponding to a first particular node of said plurality of computing nodes;
• generating a second unique identifier particularly corresponding to a second particular node of said plurality of computing nodes;
• storing task information in said shared data storage, said task information indicative of a plurality of computing tasks each available to be completed by one of said plurality of computing nodes;
• periodically updating node information in said shared data storage with new information associated with said first unique identifier;
• accessing said shared data storage at a first time;
• identifying a most recent update to said node information associated with said first unique identifier in said shared data storage;
• determining whether said most recent update to said node information associated with said first unique identifier in said shared data storage occurred more than a threshold amount of time prior to said first time;
• concluding, when said most recent update to said node information associated with said first unique identifier in said shared data storage occurred more than a threshold amount of time prior to said first time, that a first task identified by said task information as being processed by said first particular node is no longer being processed by said first particular node;
• and
• processing said first task that is concluded to be no longer being processed by said first particular node with a second particular node of said plurality of computing nodes;
• and wherein
• said task information includes a plurality of records, each record including a first field including information identifying a particular task,
• a second field including information identifying a particular status associated with said particular task identified by said information of said first field, and
• a third field including information identifying a particular node of said plurality of computing nodes associated with said particular task identified by said information of said first field.

A method for identifying failed computing nodes, said method comprising:

• providing a plurality of computing nodes, each computing node of said plurality of computing nodes having access to at least one hardware processor and to shared data storage;
• generating a first unique identifier particularly corresponding to a first particular node of said plurality of computing nodes;
• generating a second unique identifier particularly corresponding to a second particular node of said plurality of computing nodes;
• storing task information in said shared data storage, said task information indicative of a plurality of computing tasks each available to be completed by one of said plurality of computing nodes;
• periodically updating node information in said shared data storage with new information associated with said first unique identifier;
• accessing said shared data storage at a first time;
• identifying a most recent update to said node information associated with said first unique identifier in said shared data storage;
• determining whether said most recent update to said node information associated with said first unique identifier in said shared data storage occurred more than a threshold amount of time prior to said first time;
• concluding, when said most recent update to said node information associated with said first unique identifier in said shared data storage occurred more than a threshold amount of time prior to said first time, that a first task identified by said task information as being processed by said first particular node is no longer being processed by said first particular node;
• and
• processing said first task that is concluded to be no longer being processed by said first particular node with a second particular node of said plurality of computing nodes;
• and
• said step of periodically updating said node information includes updating said node information with a new time stamp associated with said first particular node at a predetermined interval.

A method for identifying failed computing nodes, said method comprising:

• providing a plurality of computing nodes, each computing node of said plurality of computing nodes having access to at least one hardware processor and to shared data storage;
• generating a first unique identifier particularly corresponding to a first particular node of said plurality of computing nodes;
• generating a second unique identifier particularly corresponding to a second particular node of said plurality of computing nodes;
• storing task information in said shared data storage, said task information indicative of a plurality of computing tasks each available to be completed by one of said plurality of computing nodes;
• periodically updating node information in said shared data storage with new information associated with said first unique identifier;
• accessing said shared data storage at a first time;
• identifying a most recent update to said node information associated with said first unique identifier in said shared data storage;
• determining whether said most recent update to said node information associated with said first unique identifier in said shared data storage occurred more than a threshold amount of time prior to said first time;
• concluding, when said most recent update to said node information associated with said first unique identifier in said shared data storage occurred more than a threshold amount of time prior to said first time, that a first task identified by said task information as being processed by said first particular node is no longer being processed by said first particular node;
• processing said first task that is concluded to be no longer being processed by said first particular node with a second particular node of said plurality of computing nodes;
• after said step of concluding, generating a new unique identifier particularly corresponding to said first particular node of said plurality of computing nodes;
• and
• updating said node information with said new unique identifier;
• and wherein said first unique identifier includes information identifying said first particular node, and time information indicative of a first particular time;
• and
• said new unique identifier particularly corresponding to said first particular node includes said information identifying said first particular node and time information indicative of a second particular time different than said first particular time.

A multi-node computing system, comprising:

• a plurality of computing nodes, each computing node of said plurality of computing nodes having access to associated node memory, for storing data and code, and at least one hardware processor configured to execute said code, said code including a set of native instructions configured to cause said at least one hardware processor to perform a corresponding set of native operations when executed by said at least one hardware processor;
• and
• shared data storage accessible to each node of said plurality of computing nodes, said shared data storage including task information and node information, said task information indicative of a plurality of tasks each available to be processed by one of said nodes, said node information indicative of a status of each of said nodes;
• and wherein
• said data and said code associated with a first one of said nodes include
• a first subset of said set of native instructions configured to generate a first unique identifier particularly corresponding to said associated first node,
• a second subset of said set of native instructions configured to periodically update said node information in said shared data storage with new information associated with said first unique identifier of said first node,
• a third subset of said set of native instructions configured to determine whether a first task, previously identified by said task information as being processed by another node, is available for processing by said first node, and
• a fourth subset of said set of native instructions configured to process said first task when it is determined to be available;
• and
• said second subset of said set of native instructions is configured to periodically update said node information by storing new time information associated with said first node by said first unique identifier in said node information.

A multi-node computing system, comprising:

• a plurality of computing nodes, each computing node of said plurality of computing nodes having access to associated node memory, for storing data and code, and at least one hardware processor configured to execute said code, said code including a set of native instructions configured to cause said at least one hardware processor to perform a corresponding set of native operations when executed by said at least one hardware processor;
• and
• shared data storage accessible to each node of said plurality of computing nodes, said shared data storage including task information and node information, said task information indicative of a plurality of tasks each available to be processed by one of said nodes, said node information indicative of a status of each of said nodes;
• and wherein
• said data and said code associated with a first one of said nodes include
• a first subset of said set of native instructions configured to generate a first unique identifier particularly corresponding to said associated first node,
• a second subset of said set of native instructions configured to periodically update said node information in said shared data storage with new information associated with said first unique identifier of said first node,
• a third subset of said set of native instructions configured to determine whether a first task, previously identified by said task information as being processed by another node, is available for processing by said first node, and
• a fourth subset of said set of native instructions configured to process said first task when it is determined to be available;
• said first unique identifier includes information identifying said first node, and time information indicative of a first time;
• upon restart of said first node, said first subset of said set of native instructions is configured to generate a new unique identifier particularly corresponding to said associated first node;
• and
• said new unique identifier includes said information identifying said first node and includes new time information indicative of a second time different than said first time.