Machine learning and optimization techniques for job scheduling

An apparatus comprising:

• a memory;
• and
• a hardware processor communicatively coupled to the memory, the hardware processor configured to:
• train a recurrent neural network by monitoring data in a memory of a first server as the first server executes jobs and by determining an amount of computing resources used by the first server while executing the jobs;
• apply the recurrent neural network to starting data in the memory of the first server to predict an amount of computing resources that the first server will use when executing a first future job;
• assign the first future job to execute on the first server based on the predicted amount of computing resources;
• in response to determining that execution of the first future job did not meet a performance criterion, make a change to the first server of the computing system;
• in response to determining that execution of the first future job after make the change to the first server did meet the performance criterion, further train the recurrent neural network using a reinforcement learning technique and based on the change;
• apply the recurrent neural network to determine that the change should be made to a second server of the computing system before the second server executes a second future job and in response, make the change to the second server before the second server executes the second future job;
• and
• assign the second future job to execute on the second server.

A computer program product for job scheduling, the computer program product comprising a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to:

• train a recurrent neural network by monitoring data in a memory of a first server as the first server executes jobs and by determining an amount of computing resources used by the first server while executing the jobs;
• apply the recurrent neural network to starting data in the memory to predict an amount of computing resources that the first server will use when executing a first future job;
• assign the first future job to execute on the first server based on the predicted amount of computing resources;
• in response to determining that execution of the first future job did not meet a performance criterion, make a change to the first server of the computing system;
• in response to determining that execution of the first future job after make the change to the first server did meet the performance criterion, further train the recurrent neural network using a reinforcement learning technique and based on the change;
• apply the recurrent neural network to determine that the change should be made to a second server of the computing system before the second server executes a second future job and in response, make the change to the second server before the second server executes the second future job;
• and
• assign the second future job to execute on the second server.