Determining storage tiers for placement of data sets during execution of tasks in a workflow

A computer program product for assigning tasks to storage tiers to store data sets processed by the tasks, wherein the computer program product comprises a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause operations, the operations comprising:

• processing a representation of a workflow execution pattern of tasks for a job indicating a dependency of the tasks and data sets operated on by the tasks;
• determining an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern, wherein a higher performing storage tier includes faster access storage devices than a relatively lower performing storage tier;
• determining whether a current storage tier on which a subject data set of the data sets operated on by a completed task is located is a higher performing storage tier than a storage tier assigned to an additional task currently operating on the subject data set operated on by the completed task;
• and
• scheduling a moving of the subject data set operated on by the completed task to the storage tier assigned to the additional task in response to determining that the storage tier assigned to the completed task is the higher performing storage tier than the storage tier assigned to the completed task.

A system coupled to a plurality of storage tiers, comprising:

• a plurality of computational nodes;
• and
• a computer readable storage medium having program instructions that when executed by the computational nodes perform operations, the operations comprising:
• processing a representation of a workflow execution pattern of tasks for a job indicating a dependency of the tasks and data sets operated on by the tasks;
• determining an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern, wherein a higher performing storage tier includes faster access storage devices than a relatively lower performing storage tier;
• determining whether a subject data set of the data sets to be operated on by a next task to be started is being operated on by another task on a current storage tier;
• determining whether an assigned storage tier indicated in the assignment for the next task is a higher performing storage tier than the current storage tier in response to determining that the subject data set is being operated on by another task;
• and
• scheduling a moving of the subject data set to the assigned storage tier in response to determining that the assigned storage tier is higher performing than the current storage tier, and wherein the subject data set is scheduled to remain on the current storage tier in response to determining that the assigned storage tier is not higher performing than the current storage tier.

A system coupled to a plurality of storage tiers, comprising:

• a plurality of computational nodes;
• and
• a computer readable storage medium having program instructions that when executed by the computational nodes perform operations, the operations comprising:
• processing a representation of a workflow execution pattern of tasks for a job indicating a dependency of the tasks and data sets operated on by the tasks;
• determining an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern, wherein a higher performing storage tier includes faster access storage devices than a relatively lower performing storage tier;
• determining whether a current storage tier on which a subject data set of the data sets operated on by a completed task is located is a higher performing storage tier than a storage tier assigned to an additional task currently operating on the subject data set operated on by the completed task;
• and
• scheduling a moving of the subject data set operated on by the completed task to the storage tier assigned to the additional task in response to determining that the storage tier assigned to the completed task is the higher performing storage tier than the storage tier assigned to the completed task.

A method for assigning tasks to storage tiers to store data sets processed by the tasks, comprising:

• processing a representation of a workflow execution pattern of tasks for a job indicating a dependency of the tasks and data sets operated on by the tasks;
• determining an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern, wherein a higher performing storage tier includes faster access storage devices than a relatively lower performing storage tier;
• determining whether a subject data set of the data sets to be operated on by a next task to be started is being operated on by another task on a current storage tier;
• determining whether an assigned storage tier indicated in the assignment for the next task is a higher performing storage tier than the current storage tier in response to determining that the subject data set is being operated on by another task;
• and
• scheduling a moving of the subject data set to the assigned storage tier in response to determining that the assigned storage tier is higher performing than the current storage tier, and wherein the subject data set is scheduled to remain on the current storage tier in response to determining that the assigned storage tier is not higher performing than the current storage tier.

A method for assigning tasks to storage tiers to store data sets processed by the tasks, comprising:

• processing a representation of a workflow execution pattern of tasks for a job indicating a dependency of the tasks and data sets operated on by the tasks;
• determining an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern, wherein a higher performing storage tier includes faster access storage devices than a relatively lower performing storage tier;
• determining whether a current storage tier on which a subject data set of the data sets operated on by a completed task is located is a higher performing storage tier than a storage tier assigned to an additional task currently operating on the subject data set operated on by the completed task;
• and
• scheduling a moving of the subject data set operated on by the completed task to the storage tier assigned to the additional task in response to determining that the storage tier assigned to the completed task is the higher performing storage tier than the storage tier assigned to the completed task.