Dynamic kernel slicing for VGPU sharing in serverless computing systems

A method for dynamic kernel slicing in a serverless computing service, comprising:

• receiving a request for execution of program code in the serverless computing service in which a plurality of virtual graphics processing units (vGPUs) are used in the execution of the program code on an underlying physical graphics processing unit (GPU);
• in response to receipt of the request for execution of the program code, determining a slice size to partition a compute kernel of the program code into a plurality of sub-kernels for concurrent execution by the vGPUs, the slice size being determined for individual ones of the sub-kernels using an optimization function in accordance with a scheduling policy;
• determining an execution schedule for executing the individual ones of the sub-kernels on the vGPUs in accordance with the scheduling policy;
• and
• executing the sub-kernels on the vGPUs as partitioned in accordance with the execution schedule.

A non-transitory computer-readable medium comprising program instructions for dynamic kernel slicing in a serverless computing service that, when executed by at least one computing device, direct the at least one computing device to:

• receive a request for execution of program code in the serverless computing service in which a plurality of virtual graphics processing units (vGPUs) are used in the execution of the program code on an underlying physical graphics processing unit (GPU);
• in response to receipt of the request for execution of the program code, determine a slice size to partition a compute kernel of the program code into a plurality of sub-kernels for concurrent execution by the vGPUs, the slice size being determined for individual ones of the sub-kernels using an optimization function in accordance with a scheduling policy;
• determine an execution schedule for executing the individual ones of the sub-kernels on the vGPUs in accordance with the scheduling policy;
• and
• execute the sub-kernels on the vGPUs as partitioned in accordance with the execution schedule.