Managing instances of serverless functions in a cloud computing system

A computing system having a memory having computer readable instructions and one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:

• obtaining a service level objective for a serverless function, wherein the service level objective specifies a maximum number of concurrent requests a compute node of a cloud computing system can process for the serverless function;
• obtaining a command queue length for a graphical processing unit (GPU) disposed on each of a plurality of compute nodes in the cloud computing system;
• obtaining a request queue length of the serverless function;
• calculating a number of instances of the serverless function to deploy in the cloud computing system, wherein the number of instances is determined based on the service level objective and the request queue length of the serverless function;
• identifying, based at least in part on the command queue length for the GPU of each of the plurality of compute nodes, compute nodes from the plurality of compute nodes to deploy each of the number of instances of the serverless function;
• and
• creating an instance of the serverless function on each of the identified compute nodes.

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:

• obtaining a service level objective for a serverless function, wherein the service level objective specifies a maximum number of concurrent requests a compute node of a cloud computing system can process for the serverless function;
• obtaining a command queue length for a graphical processing unit (GPU) disposed on each of a plurality of compute nodes in the cloud computing system;
• obtaining a request queue length of the serverless function;
• calculating a number of instances of the serverless function to deploy in the cloud computing system, wherein the number of instances is determined based on the service level objective and the request queue length of the serverless function;
• identifying, based at least in part on the command queue length for the GPU of each of the plurality of compute nodes, compute nodes from the plurality of compute nodes to deploy each of the number of instances of the serverless function;
• and
• creating an instance of the serverless function on each of the identified compute nodes.