Proactive cluster compute node migration at next checkpoint of cluster cluster upon predicted node failure

A system comprising:

• a cluster of compute nodes, including active compute nodes to distributively execute an application in parallel, and a spare compute node;
• checkpoint logic to regularly checkpoint the active compute nodes;
• and
• failure prediction and node migration logic to:
• periodically predict a likelihood of failure of the active compute nodes using machine learning, and
• responsive to the likelihood of failure of a given active compute node exceeding a threshold, migrate the selected given active compute node to the spare compute node after the checkpoint logic next regularly checkpoints the active compute node,
• wherein the failure prediction and node migration logic operates independently of the checkpoint logic.

A non-transitory computer-readable data storage medium storing instructions executable by a processor to:

• predict a likelihood of active compute nodes of a cluster that are distributively executing an application in parallel, based on a software event selected from a group of software events comprising:
• a last number of ranks of the application that reach a synchronization barrier during execution of the application;
• a last number of ranks of the application that provide state data when the checkpoints are being taken;
• performance results of a software test periodically executed on the active compute nodes representatively revealing hardware problems of the active compute nodes;
• shadow execution on a second spare compute node of tasks regarding the application that a selected active compute node is executing;
• and
• responsive to the likelihood of failure of a given active compute node exceeding a threshold, proactively migrate the given active compute node to the spare compute node of the cluster at a next checkpoint of the active compute nodes.