Implicit status in many task computing

A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause at least one processor to perform operations comprising:

• receive, at the at least one processor and from a requesting device via a network, a first request for status of a requested performance of a job flow, wherein:
• the job flow is defined in a job flow definition that specifies a set of tasks to be performed via execution of a corresponding set of task routines during a performance of the job flow, and that specifies data dependencies among the set of tasks;
• store, within a job queue, a job performance status request message comprising a job flow instance identifier assigned to the requested performance;
• within a performance container, execute instructions of an instance of a performance routine to cause the at least one processor to perform operations comprising:
• derive an order of performance of the set of tasks based on indications in the job flow definition of dependencies among the set of tasks;
• store, within a task queue, multiple task routine execution request messages in an order that is based on the derived order of performance and on corresponding task completion messages stored within the task queue, to cause at least a subset of the set of task routines to be executed in the derived order of performance to cause at least a corresponding subset of the set of tasks to be performed in the derived order of performance to perform the job flow, resulting in the storage of the corresponding task completion messages;
• and
• in response to the storage of the job performance status request message within the job queue, store, within the job queue, a job performance status indication comprising a set of task performance statuses for the set of tasks that are based on the task completion messages stored within the task queue;
• analyze indications in the job flow definition of dependencies among the set of tasks to identify a set of sub flows within the job flow based on branches in the job flow, wherein each sub flow comprises multiple tasks of the set of tasks that are to be performed sequentially;
• based on which tasks of the set of tasks are included within each sub flow, correlate each of the task performance statuses in the job performance status indication to a corresponding sub flow performance status;
• reduce a size of the job performance status indication from the size at which the job performance status indication is stored in the job queue by performing operations comprising:
• for each sub flow in which all tasks of the sub flow have been completed, replace the corresponding task performance statuses with the corresponding sub flow performance status of completed;
• and
• for each sub flow in which no task has been performed and in which no task is currently being performed, replace the corresponding task performance statuses with the corresponding sub flow performance status of not yet executed;
• and
• following the reduction in size of the job performance status indication, transmit the job performance status indication to the requesting device.

A computer-implemented method comprising:

• receiving, by at least one processor and from a requesting device via a network, a first request for status of a requested performance of a job flow, wherein:
• the job flow is defined in a job flow definition that specifies a set of tasks to be performed via execution of a corresponding set of task routines during a performance of the job flow, and that specifies data dependencies among the set of tasks;
• storing, within a job queue, a job performance status request message comprising a job flow instance identifier assigned to the requested performance;
• within a performance container, executing, by the at least one processor, instructions of an instance of a performance routine to cause the at least one processor to perform operations comprising:
• deriving, by the at least one processor, an order of performance of the set of tasks based on indications in the job flow definition of dependencies among the set of tasks;
• storing, within a task queue, multiple task routine execution request messages in an order that is based on the derived order of performance and on corresponding task completion messages stored within the task queue, to cause at least a subset of the set of task routines to be executed in the derived order of performance to cause, by the at least one processor, at least a corresponding subset of the set of tasks to be performed in the derived order of performance to perform the job flow, resulting in the storage of the corresponding task completion messages;
• and
• in response to the storage of the job performance status request message within the job queue, storing, within the job queue, a job performance status indication comprising a set of task performance statuses for the set of tasks that are based on the task completion messages stored within the task queue;
• analyzing, by the at least one processor, indications in the job flow definition of dependencies among the set of tasks to identify a set of sub flows within the job flow based on branches in the job flow, wherein each sub flow comprises multiple tasks of the set of tasks that are to be performed sequentially;
• based on which tasks of the set of tasks are included within each sub flow, correlating, by the at least one processor, each of the task performance statuses in the job performance status indication to a corresponding sub flow performance status;
• reducing, by the at least one processor, a size of the job performance status indication from the size at which the job performance status indication is stored in the job queue by performing operations comprising:
• for each sub flow in which all tasks of the sub flow have been completed, replacing the corresponding task performance statuses with the corresponding sub flow performance status of completed;
• and
• for each sub flow in which no task has been performed and in which no task is currently being performed, replacing the corresponding task performance statuses with the corresponding sub flow performance status of not yet executed;
• and
• following the reduction in size of the job performance status indication, transmitting, from the at least one processor, the job performance status indication to the requesting device.