Method and system for optimizing GPU utilization
AssigneeProphetStor Data Services, Inc.
InventorsWen-Shyen Chen, Ming-Jye Sheu, Henry Hong-Yi Tzeng, Sheng-Lin Wu
Life of this patent
Estimating the term…
Abstract
The present invention provides a method for optimizing graphics-processing unit (GPU) utilization and a system thereof. The method includes the following steps: receiving an application workload which is to be executed on the GPU; predicting GPU resource requirements for the application workload; scheduling the application workload according to the prediction of the GPU resource requirements; dynamically allocating and deallocating GPU resources based on the prediction of the GPU resource requirements for the application workload; and executing the application workload on the GPU.
Claims19
A method for optimizing graphics-processing unit (GPU) utilization, comprising the following steps:
- receiving by an application unit an application workload which is to be executed on the GPU based on a user request;
- a GPU infrastructure, including a plurality of GPU clusters, a virtual machine (VM)/Kubernetes platform, connected to the application unit and the GPU infrastructure, for managing the plurality of GPU clusters in the GPU infrastructure and
- predicting GPU resource requirements for the application workload by using a managing and predicting unit, connected to the application unit, the GPU infrastructure, and the VM/Kubernetes platform;
- providing scheduling the application workload according to the prediction of the GPU resource requirements to the VM/Kubernetes platform;
- dynamically allocating and deallocating GPU resources based on the prediction of the GPU resource requirements for the application workload via the VM/Kubernetes platform;
- and
- executing the application workload on the GPU;
- wherein the GPU resource requirements for the application workload are predicted by the following steps:
- a) deploy and install an application on multiple nodes;
- b) periodically collect workloads of the application and resource usage in the nodes, and calculate correlation values of resource usage for the application and the sub-application;
- c) use a time series model to predict the application's workload at a future time point (T+1) based on current time point (T) and past time points (T−1, T−2, . . . T−n) and identify resources with high correlation values above a specified threshold;
- and
- d) develop a predictive model for resource usage which uses past resource usage data and predicted application workload at time point T+1 to estimate resource usage increments for the identified resource from step c.
A system for optimizing graphics-processing unit (GPU) utilization, comprising:
- an application unit, for receiving an application workload which is to be executed on the GPU based on a user request;
- a GPU infrastructure, including a plurality of GPU clusters;
- a virtual machine (VM)/Kubernetes platform, connected to the application unit and the GPU infrastructure, for managing the plurality of GPU clusters in the GPU infrastructure and scheduling the application workload which is to be executed on the GPU;
- and
- a managing and predicting unit, connected to the application unit, the GPU infrastructure, and the VM/Kubernetes platform, for predicting GPU resource requirements for the application workload;
- providing scheduling of the application workload according to the prediction of the GPU resource requirements to the VM/Kubernetes platform;
- and
- dynamically allocating and deallocating GPU resources based on the prediction of the GPU resource requirements for the application workload via the VM/Kubernetes platform;
- wherein the GPU resource requirements for the application workload are predicted by the following steps:
- a) deploy and install an application on multiple nodes;
- b) periodically collect workloads of the application and resource usage in the nodes, and calculate correlation values of resource usage for the application and the sub-application;
- c) use a time series model to predict the application's workload at a future time point (T+1) based on current time point (T) and past time points (T−1, T−2, . . . T−n) and identify resources with high correlation values above a specified threshold;
- and
- d) develop a predictive model for resource usage which uses past resource usage data and predicted application workload at time point T+1 to estimate resource usage increments for the identified resource from step c.
Dependencies are inferred from the claim text.