Coordinated, topology-aware CPU-GPU-memory scheduling for containerized workloads

A computer program product for workload scheduling, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:

• analyzing:
• a resource scheduling requirement for processes of a workload involving computer accelerators including a Graphics Processing Unit (GPU) as a discrete type of a computing resource;
• a relationship, a capacity, and a processing connection type between:
• the GPU to a Central Processing Unit (CPU) and network connection;
• computer accelerators and second network connection between the computer accelerators, the network connection between the computer accelerators and the CPU being different than, the second network connection between the computer accelerators;
• and
• a location of the computer accelerators in sockets in relation to each other and to the CPU;
• creating feasible resources having a specific kernel implementation for a specific type of the GPU based on static characteristics and topology configuration of the resources for the processes of the workload;
• selecting an available resource of the feasible resources to assign to the workload based on the resource scheduling requirement, the capacity, and according to assignment preference rules, the assignment preference rules being applied based on the relationship and processing connection type for performance gain;
• implementing a feedback step for monitoring an actual workload of the processes after the selecting selects the available resource;
• and
• assigning an additional processes to the GPU to be performed concurrently with the workload when the actual workload is determined less than a threshold amount by the feedback step.

A workload scheduling system, said system comprising:

• a processor;
• and
• a memory, the memory storing instructions to cause the processor to perform:
• analyzing:
• a resource scheduling requirement for processes of a workload involving computer accelerators including a Graphics Processing Unit (GPU) as a discrete type of a computing resource;
• a relationship, a capacity, and a processing connection type between:
• the GPU to a Central Processing Unit (CPU) and network connection;
• computing accelerators and second network connection between the computer accelerators, the network connection between the computer accelerators and the CPU being different than the second network connection between the computer accelerators;
• and
• a location of the computer accelerators in sockets in relation to each other and to the CPU;
• creating feasible resources having a specific kernel implementation for a specific type of the GPU based on static characteristics and topology configuration of the resources for the processes of the workload;
• selecting an available resource of the feasible resources to assign to the workload based on the resource scheduling requirement, the capacity, and according to assignment preference rules, the assignment preference rules being applied based on the relationship and processing connection type for performance gain;
• implementing a feedback step for monitoring an actual workload of the processes after the selecting selects the available resource;
• and
• assigning an additional processes to the GPU to be performed concurrently with the workload when the actual workload is determined less than a threshold amount by the feedback step.

A computer-implemented workload scheduling method, the method comprising:

• collecting information on workload resource requirements for execution of a work request for processing a workload including:
• a Graphics Processing Unit (GPU) as a discrete type of a computing resource;
• a relationship, a capacity, and a processing connection type between:
• the GPU to a Central Processing Unit (CPU) and network connection;
• computer accelerators and second network connection between the computer accelerators, the network connection between the computer accelerators and the CPU being different than the second network connection between the computer accelerators;
• and
• a location of the computer accelerators in sockets in relation to each other and to the CPU;
• matching the workload resource requirements with available resources having a specific kernel implementation for a specific type of the GPU in a data center of machines;
• dispatching the workload to an allocated Graphics Processing Unit (GPU), the Central Processing Unit (CPU), and memory resources based on the matched workload resource requirements to the available resources, the capacity, and according to assignment preference rules, the assignment preference rules being, applied based on the relationship and processing connection type for performance gain;
• implementing a feedback step for monitoring an actual workload of the workload after the dispatching dispatches the available resource;
• and
• assigning an additional processes to the GPU to be performed concurrently with the workload when the actual workload is determined less than a threshold amount by the feedback step.