Fault tolerance model, methods, and apparatuses and their validation techniques

A system for fault-tolerant software release automation, the system comprising:

• one or more processing devices and
• one or more storage devices storing instructions that, when executed by the one or more processing devices, cause the one or more processing devices to:
• identify a set of software release procedures;
• model, using a fault-tolerant release automation library, the set of software release procedures and their dependencies into a nested directed graph of workflow components, where at least one component represents at least one software release procedure from the set of software release procedures;
• create a release program using the modeled software release procedures;
• running an end to end test of the release program to verify configuration files used in a release;
• changing resource labels within the release program for production resources not available in a testing environment to create relative paths;
• automatically replicating a runtime instance for testing purposes using non-production data using the relative paths created by the changed resource labels, the changed resource labels including prefix or postfix labels, and
• running the created release program, wherein each workflow component includes at least one operation performed by the release program,
• wherein the nested directed graph includes a plurality of vertices and directed arcs, each vertex corresponding to a workflow component and each directed arc representing a workflow component's conditional dependency of one or more preceding workflow components having successfully completed, where the conditional dependencies include both AND and OR conditions.