Hardware and parameter-aware machine learning model gpu efficiency tuning systems

A system comprising a memory, computer readable instructions, and one or more circuitry for executing the computer readable instructions, the computer readable instructions controlling the one or more circuitry to perform operations comprising:

• receive a request corresponding to a machine learning model training task;
• receive a plurality of fixed configurations comprising fixed parameters for the request;
• generate a task embedding from the plurality of fixed configurations;
• receive a plurality of dynamic configurations comprising variable parameters for the request, the variable parameters comprising tunable hyperparameters;
• train a prediction module on training data comprising known dynamic and fixed configurations and, for each combination of a known dynamic configuration and a known fixed configuration, a respective model utilization score;
• generate, from the prediction module, a plurality of model utilization scores for a plurality of respective candidate configurations sampled from the plurality of dynamic configurations, wherein generating the model utilization score comprises:
• sample a candidate configuration from the plurality of dynamic configurations, the candidate configuration comprising candidate parameter values;
• generate a candidate configuration embedding from the respective sampled candidate configuration;
• and
• generate, responsive to inputting the candidate configuration embedding and the task embedding to the prediction module, a model utilization score for the respective sampled candidate configuration;
• and
• returning, responsive to receiving the request, a response comprising a training configuration for the machine learning model training task, the training configuration comprising a respective sampled candidate configuration having a model utilization score satisfying a predetermined threshold.

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more circuitry to cause the one or more circuitry to perform operations comprising:

• receive a request corresponding to a machine learning model training task;
• receive a plurality of fixed configurations comprising fixed parameters for the request;
• generate a task embedding from the plurality of fixed configurations;
• receive a plurality of dynamic configurations comprising variable parameters for the request, the variable parameters comprising tunable hyperparameters;
• train a prediction module on training data comprising known dynamic and fixed configurations and, for each combination of a known dynamic configuration and a known fixed configuration, a respective model utilization score;
• generate, from the prediction module, a plurality of model utilization scores for a plurality of respective candidate configurations sampled from the plurality of dynamic configurations, wherein generating the model utilization score comprises:
• sample a candidate configuration from the plurality of dynamic configurations, the candidate configuration comprising candidate parameter values;
• generate a candidate configuration embedding from the respective sampled candidate configuration;
• and
• generate, responsive to inputting the candidate configuration embedding and the task embedding to the prediction module, a model utilization score for the respective sampled candidate configuration;
• and
• return, responsive to receiving the request, a response comprising a training configuration for the machine learning model training task, the training configuration comprising a respective sampled candidate configuration having a model utilization score satisfying a predetermined threshold.