Job processing method and apparatus, computer device, and storage medium
AssigneeHuawei Technologies Co., Ltd.
InventorsShoufeng Hu, Bo Zhu, Jin Guo, Dong Dong
Life of this patent
Estimating the term…
Abstract
A job processing method is provided. In this method, after receiving a to-be-processed job, a scheduling node in a high-performance computing system predicts, based on a category of the job, a quantity of resources consumed in real time in a process of executing the job. Because quantities of resources consumed in real time in a process of executing jobs of a same category are close, compared with a specified resource quantity, a predicted resource quantity is closer to a quantity of resources actually consumed by the job.
Claims19
A method, performed by a scheduling node in a high-performance computing system, wherein the method comprises:
- receiving a mode adjustment instruction, wherein the mode adjustment instruction instructs to enable a prediction mode, and provide, in the prediction mode, a function of predicting a quantity of resources required by a job;
- enabling the prediction mode according to the mode adjustment instruction;
- receiving a job to be processed by the high-performance computing system;
- determining a category of the job;
- predicting a predicted resource quantity of the job based on the category of the job, wherein the predicted resource quantity is a predicted quantity of resources to be consumed in real time while executing the job, wherein predicting the predicted resource quantity of the job based on the category of the job comprises predicting, in the prediction mode based on a reference resource quantity corresponding to the category of the job, a quantity of resources to be consumed in real time while executing the job, to obtain the predicted resource quantity, wherein the reference resource quantity is obtained based on a quantity of resources actually consumed by a historical job of a same category as the category of the job;
- and
- scheduling, based on the predicted resource quantity, a computing node in the high-performance computing system to execute the job.
A device, comprising:
- a non-transitory memory storage comprising instructions;
- and
- one or more processors in communication with the non-transitory memory storage, wherein the one or more processors execute the instructions for:
- receiving a mode adjustment instruction, wherein the mode adjustment instruction instructs to enable a prediction mode, and provide, in the prediction mode, a function of predicting a quantity of resources required by a job;
- enabling the prediction mode according to the mode adjustment instruction;
- receiving a job to be processed by a high-performance computing system;
- determining a category of the job;
- predicting a predicted resource quantity of the job based on the category of the job, wherein the predicted resource quantity is a predicted quantity of resources to be consumed in real time while executing the job, wherein predicting the predicted resource quantity of the job based on the category of the job comprises predicting, in the prediction mode based on a reference resource quantity corresponding to the category of the job, a quantity of resources to be consumed in real time while executing the job, to obtain the predicted resource quantity, wherein the reference resource quantity is obtained based on a quantity of resources actually consumed by a historical job of a same category as the category of the job;
- and
- scheduling, based on the predicted resource quantity, a computing node in the high-performance computing system to execute the job.
A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores at least one piece of program code, and the at least one piece of program code is read by one or more processors, to enable a computer device to:
- receive a mode adjustment instruction, wherein the mode adjustment instruction instructs to enable a prediction mode, and provide, in the prediction mode, a function of predicting a quantity of resources required by a job;
- enable the prediction mode according to the mode adjustment instruction;
- receive a job to be processed by a high-performance computing system;
- determine a category of the job;
- predict a predicted resource quantity of the job based on the category of the job, wherein the predicted resource quantity is a predicted quantity of resources to be consumed in real time while executing the job, wherein predicting the predicted resource quantity of the job based on the category of the job comprises predicting, in the prediction mode based on a reference resource quantity corresponding to the category of the job, a quantity of resources to be consumed in real time while executing the job, to obtain the predicted resource quantity, wherein the reference resource quantity is obtained based on a quantity of resources actually consumed by a historical job of a same category as the category of the job;
- and
- schedule, based on the predicted resource quantity, a computing node in the high-performance computing system to execute the job.
Dependencies are inferred from the claim text.