Real-time monitoring of job resource consumption and prediction of resource deficiency based on future availability

A system for providing real time job resource monitoring of a computer system, comprising:

• one or more processors;
• a resource estimator executing on the one or more processors configured for analyzing input data of a job provided to the computer system and for estimating a total resource requirement representing adequate levels of processing resources in the computer system needed for successful execution of the job, the resource estimator further configured for obtaining a series of application performance benchmarks against the computer system, for determining a resource consumption rate of the job based on the application performance benchmarks, and for estimating a total amount of time required to execute the job on the computer system;
• a resource monitor executing on the one or more processors configured for monitoring adequate processing resources of the computer system throughout execution of the job within the computer system, by:
• measuring cumulative resource consumption for the job and available resource levels in the computer system;
• factoring remaining execution time from the total amount of time required to execute the job on the computer system;
• estimating future resource requirements of the execution of the job based on the remaining execution time, the total resource requirements of the job and the cumulative resource consumption for the job;
• and
• comparing the future resource requirements of the execution of the job with the available resource levels in the computer system that are projected to exist in the future to predict any resource deficiency;
• and
• a resource manager executing on the one or more processors configured for implementing one or more corrective actions to prevent failure of the job execution on the computer system responsive to prediction of a resource deficiency, wherein the resource monitor is further configured for monitoring adequate processing resources of the computer system at a predefined interval throughout execution of the job within the computer system, the predetermined interval correlating to one or both of the input data's size and the total resource requirements of the job.