Automated ETL resource provisioner

A system, comprising:

• a processor;
• a computer readable memory in circuit communication with the processor;
• and
• a computer readable storage medium in circuit communication with the processor;
• wherein the processor executes program instructions stored on the computer-readable storage medium via the computer readable memory and thereby:
• monitors each of a plurality of operator steps carried out to accomplish a structured English query language command job for their respective usages of available extract, transform and load cloud resources during the performance of each of said operator job steps, wherein the available cloud resources are selected from the group consisting of central processing unit, memory, storage and input/output device resources;
• logs the respective usage data in association with respective ones of the operator steps that generate the usage data;
• compares the logged usage data to historic data for matching steps to determine under-utilization or over-utilization of the cloud resources for each of the steps as a function of amounts of differences in values of the compared data relative to the historic data for the matching steps, comprising determining the over-utilization of the cloud resources in response to determining that a first central processing unit of a plurality of central processing units is operating at its maximum capacity in executing a first of the monitored steps, and determining the under-utilization of the cloud resources in response to determining that a second central processing unit of the plurality of central processing units is operating at a minimal load in executing a second of the monitored steps;
• and
• optimizes an operating environment for subsequent executions of the monitored plurality of operator steps by adjusting a value selected from the group consisting of amount, type or speed, with respect to central processing units, memories or storage devices of cloud resources that are associated with the operator steps to correspondingly reduce the amounts of differences in values of the compared data relative to the historic data for the matching steps, by:
• allocating an additional central processing unit for the subsequent executions of the first monitored step in response to the determining that the first central processing unit is operating at its maximum capacity in executing the first monitored step;
• and
• dropping an allocation of the second central processing unit for the subsequent executions of the second monitored step in response to the determining that the second central processing unit is operating at the minimal load in executing the second monitored step.

A computer program product for the automated provisioning of cloud resources for extract, transform and load job execution as a function of logged data, the computer program product comprising:

• a computer readable storage medium having computer readable program code embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the computer readable program code comprising instructions for execution by a processor that cause the processor to:
• monitor each of a plurality of operator steps carried out to accomplish a structured English query language command job for their respective usages of available extract, transform and load cloud resources during the performance of each of said operator job steps, wherein the available cloud resources are selected from the group consisting of central processing unit, memory, storage and input/output device resources;
• log the respective usage data in association with respective ones of the operator steps that generate the usage data;
• compare the logged usage data to historic data for matching steps to determine under-utilization or over-utilization of the cloud resources for each of the steps as a function of amounts of differences in values of the compared data relative to the historic data for the matching steps, comprising determining the over-utilization of the cloud resources in response to determining that a first central processing unit of a plurality of central processing units is operating at its maximum capacity in executing a first of the monitored steps, and determining the under-utilization of the cloud resources in response to determining that a second central processing unit of the plurality of central processing units is operating at a minimal load in executing a second of the monitored steps;
• and
• optimize an operating environment for subsequent executions of the monitored plurality of operator steps by adjusting a value selected from the group consisting of amount, type or speed, with respect to central processing units, memories or storage devices of cloud resources that are associated with the operator steps to correspondingly reduce the amounts of differences in values of the compared data relative to the historic data for the matching steps, by:
• allocating an additional central processing unit for the subsequent executions of the first monitored step in response to the determining that the first central processing unit is operating at its maximum capacity in executing the first monitored step;
• and
• dropping an allocation of the second central processing unit for the subsequent executions of the second monitored step in response to the determining that the second central processing unit is operating at the minimal load in executing the second monitored step.