Apparatus and method for fault-tolerant computing

A computer system for fault tolerant computing comprising:

• at least one processor for executing user level processes;
• a first user level process executing on said at least one processor;
• a user level daemon process executing on said at least one processor;
• means for providing a registration message specifying said first user level process to said user level daemon process, said user level daemon process being responsive to said registration message by initiating observation of said first user level process to determine whether said first user level process is unable to continue execution and for restarting said first user level process when said first user level process is observed to be unable to continue execution.

A computer system for fault tolerant computing comprising:

• a plurality of nodes, each of said nodes comprising at least one processor for executing user level processes;
• a first user level process executing in a first one of said nodes;
• a first user level daemon process executing in a second one of said nodes for actively polling said first one of said nodes for determining whether said first one of said nodes is inoperable, and for restarting said first user level process on said second one of said nodes when said first user level daemon process determines that said first one of said nodes is inoperable;
• a second user process executing in said second one of said nodes, wherein said first user level daemon further observes whether said second user process is unable to continue execution and restarts said second user level process on said second one of said nodes when said second user level process is observed to be unable to continue;
• a second one of said user level daemon processes executing on said first one of said nodes, said second one of said user level daemon processes copies state information from said first user level process to said first user level daemon process, and said first user level daemon process employs said state information in restarting execution of said first user level process;
• and
• computer program code executed by said first user level process, said computer program code comprising a first fault tolerant library routine which, when executed, saves said state information.

A distributed computer system for fault tolerant computing comprising:

• a plurality of nodes, each of said nodes comprising:
• at least one processor for executing user level processes;
• a user level daemon process executing on said at least one processor;
• means for providing a registration message specifying a user level process to said user level daemon, said user level daemon being responsive to said registration message by initiating observation of a specified user level process to determine whether said specified user level process is unable to continue execution and for restarting said specified user level process when said specified user level process is observed to be unable to continue executing;
• and
• wherein a first one of said user level daemon processes executing in a first one of said plurality of nodes further observes whether a second one of said plurality of nodes is operating.

A method for operation of a fault tolerant computer system, said system comprising at least one processor for executing user level processes, said method comprising the steps of:

• providing a registration message specifying a particular user level process which is executing on a processor to a user level daemon process, said user level daemon process being responsive to said registration message for performing the steps of:
• monitoring said specified user level process to determine whether said specified user level process is unable to continue executing;
• and
• restarting said specified user level process if said specified user level process is unable to continue executing.

A method for operation of a fault tolerant distributed computer system, said system comprising a plurality of nodes, each node comprising at least one processor for executing user level processes, the method comprising the steps of:

• executing a first user level daemon process on a first node, said first user level daemon process performing the steps of:
• monitoring a second node to determine whether said second node is inoperable, wherein said second node is executing a second user level process;
• and
• restarting said second user level process on said first node when it is determined that said second node is inoperable;
• executing a third user level process on said first node;
• said first user level daemon process performing the further steps of:
• monitoring said third user level process to determine whether said third user level process is unable to continue executing;
• restarting said third user level process if said third user level process is unable to continue executing;
• and
• executing a second user level daemon process on said second node;
• said second user level daemon process performs the step of:
• copying state information from said second user level process to said first user level daemon;
• said step of restarting said second user level process on said first node further comprises the step of:
• restarting said second user level process using said state information.