Interference-aware scheduling service for virtual GPU enabled systems

A method comprising:

• performing a respective workload of a plurality of workloads alone in a virtual graphics processing unit (vGPU)-enabled system to determine baseline parameters for the plurality of workloads;
• performing the respective workload co-located in the vGPU-enabled system with each other workload of the plurality of workloads to determine measured interferences for the plurality of workloads;
• training a machine learning model to predict interference based on the measured interferences and predictor variables corresponding to at least a subset of the baseline parameters for the plurality of workloads, the predictor variables being provided as inputs to the machine learning model to predict the interference, the predictor variables comprising:
• an average kernel length of a plurality of kernels of the respective workload;
• processing a workload using the machine learning model to determine a set of predicted interferences corresponding to a set of graphics processing units (GPUs), a respective one of the predicted interferences being determined based on:
• a first set of the baseline parameters comprising a first average kernel length for the workload, and a second set of the baseline parameters comprising a second average kernel length for a currently-assigned workload of a respective GPU of the set of GPUs;
• and
• assigning the workload to a particular GPU of the set of GPUs based on the particular GPU having a minimum predicted interference of the set of predicted interferences.

A non-transitory computer-readable medium comprising machine readable instructions, wherein the instructions, when executed by at least one processor, cause at least one computing device to at least:

• perform a respective workload of a plurality of workloads alone in a virtual graphics processing unit (vGPU)-enabled system to determine baseline parameters for the plurality of workloads;
• perform the respective workload co-located in the vGPU-enabled system with each other workload of the plurality of workloads to determine measured interferences for the plurality of workloads;
• train a machine learning model to predict interference based on the measured interferences and predictor variables corresponding to at least a subset of the baseline parameters for the plurality of workloads, the predictor variables being provided as inputs to the machine learning model to predict the interference, the predictor variables comprising:
• an average kernel length of a plurality of kernels of the respective workload;
• process a workload using the machine learning model to determine a set of predicted interferences corresponding to a set of graphics processing units (GPUs), a respective one of the predicted interferences being based on:
• a first set of the baseline parameters comprising a first average kernel length for the workload, and a second set of the baseline parameters comprising a second average kernel length for a currently-assigned workload of a respective GPU of the set of GPUs;
• and
• assign the workload to a particular GPU of the set of GPUs based on the particular GPU having a minimum predicted interference of the set of predicted interferences.