Dynamic accelerator scheduling and grouping for deep learning jobs in a computing cluster

A system for dynamic accelerator scheduling and grouping for deep learning jobs in a computing cluster, the system comprising:

• a plurality of accelerators;
• a memory device;
• and
• a processor executing instructions stored in the memory device, wherein the processor:
• computes an efficiency metric of each job executing in the computing cluster to generate a prioritized job queue, wherein the efficiency metric is computed based on a number of the plurality of accelerators executing a respective job, a host distribution of the number of the plurality of accelerators, and a remaining calculation time of the respective job, wherein the prioritized job queue is sorted in order of a smallest efficiency metric to a largest efficiency metric;
• and wherein the efficiency metric is updated by a scheduler at predetermined time intervals;
• generates accelerator re-grouping execution plans based on the prioritized job queue, the accelerator re-grouping execution plans associated with a target cluster topology to be achieved according to the placement of selected jobs from the prioritized job queue in relation to a location of respective ones of a plurality of accelerators within the computing cluster;
• computes a scheduling cost for each job based on the target cluster topology to generate an optimal scheduling cycle of each job to the respective ones of the plurality of accelerators;
• wherein the scheduling cost accounts for a resource adjustment cost in the scheduler and a waiting cost associated with an iteration boundary alignment;
• and
• executes one of the accelerator re-grouping execution plans to allocate the selected jobs to the respective ones of the plurality of accelerators to thereby shift the computing cluster to the target cluster topology.

A computer program product for dynamic accelerator scheduling and grouping for deep learning jobs in a computing cluster, by a processor, the computer program product embodied on a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

• an executable portion that computes an efficiency metric of each job executing in the computing cluster to generate a prioritized job queue, wherein the efficiency metric is computed based on a number of the plurality of accelerators executing a respective job, a host distribution of the number of the plurality of accelerators, and a remaining calculation time of the respective job, wherein the prioritized job queue is sorted in order of a smallest efficiency metric to a largest efficiency metric;
• and wherein the efficiency metric is updated by a scheduler at predetermined time intervals;
• an executable portion that generates accelerator re-grouping execution plans based on the prioritized job queue, the accelerator re-grouping execution plans associated with a target cluster topology to be achieved according to the placement of selected jobs from the prioritized job queue in relation to a location of respective ones of a plurality of accelerators within the computing cluster;
• an executable portion that computes a scheduling cost for each job based on the target cluster topology to generate an optimal scheduling cycle of each job to the respective ones of the plurality of accelerators;
• wherein the scheduling cost accounts for a resource adjustment cost in the scheduler and a waiting cost associated with an iteration boundary alignment;
• and
• an executable portion that executes one of the accelerator re-grouping execution plans to allocate the selected jobs to the respective ones of the plurality of accelerators to thereby shift the computing cluster to the target cluster topology.