Job merging for machine and deep learning hyperparameter tuning

A system for efficient machine and deep learning hyperparameter tuning in a distributed computing system, comprising:

• a processor executing instructions stored in a memory device;
• wherein the processor:
• collects runtime metrics of each of a plurality of training iterations to identify candidate jobs to merge during an execution phase, wherein the candidate jobs comprise hyperparameter search jobs based on a training dataset;
• identify the candidate jobs based on the collected runtime metrics according to a computed memory footprint identified for each current and previous training iteration of the plurality of training iterations of each of the candidate jobs, wherein the memory footprint is computed, for a given job of the candidate jobs and for a given iteration of the plurality of training iterations, by determining an in-graphical processing unit (in-GPU) memory consumption estimated to be required of a particular accelerator device executing the given job, and wherein determining the in-GPU memory consumption comprises computing a memory cost for the given job for the given iteration as a function of the training dataset, batch size, and model type configuration parameters of the given job;
• groups the candidate jobs into job groups, wherein the candidate jobs are initially grouped into the job groups according to the collected runtime metrics determined during the plurality of training iterations;
• and
• merges the job groups containing the candidate jobs together prior to executing the candidate jobs during the execution phase, wherein the merging of the job groups is performed within an execution engine upon receiving a merge request triggered by a scheduler determining a backlog of hyperparameter search jobs exist;
• and wherein the merging of the job groups for execution is performed for each of a plurality of accelerator devices, inclusive of the particular accelerator device, performing the execution.

A computer program product for efficient machine and deep learning hyperparameter tuning in a distributed computing system, by a processor, the computer program product embodied on a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

• an executable portion that collects runtime metrics of each of a plurality of training iterations to identify candidate jobs to merge during an execution phase, wherein the candidate jobs comprise hyperparameter search jobs based on a training dataset;
• an executable portion that identifies the candidate jobs based on the collected runtime metrics according to a computed memory footprint identified for each current and previous training iteration of the plurality of training iterations of each of the candidate jobs, wherein the memory footprint is computed, for a given job of the candidate jobs and for a given iteration of the plurality of training iterations, by determining an in-graphical processing unit (in-GPU) memory consumption estimated to be required of a particular accelerator device executing the given job, and wherein determining the in-GPU memory consumption comprises computing a memory cost for the given job for the given iteration as a function of the training dataset, batch size, and model type configuration parameters of the given job;
• an executable portion that groups the candidate jobs into job groups, wherein the candidate jobs are initially grouped into the job groups according to the collected runtime metrics determined during the plurality of training iterations;
• and
• an executable portion that merges the job groups containing the candidate jobs together prior to executing the candidate jobs during the execution phase, wherein the merging of the job groups is performed within an execution engine upon receiving a merge request triggered by a scheduler determining a backlog of hyperparameter search jobs exist;
• and wherein the merging of the job groups for execution is performed for each of a plurality of accelerator devices, inclusive of the particular accelerator device, performing the execution.