Control of hyperparameter tuning based on machine learning

A computer-implemented method, comprising:

• receiving one or more settings for a machine learning process, the settings configured to be set to a value before the machine learning process begins according to a setting establishment procedure based on one or more measurements of performance, wherein the one or more measurements of performance include an accuracy of a prediction model produced by the machine learning process and wherein the settings are configured not to be adjusted after initiation of the machine learning process;
• configure a prediction model to learn an effect of a change to the one or more settings on the one or more measurements of performance;
• adjust the settings in a first round of the setting establishment procedure;
• apply the prediction model to predict whether a second round of the setting establishment procedure is likely to result in the measurements of performance becoming closer to one or more targets;
• and
• performing the second round of the setting establishment procedure when the prediction model predicts that the measurements of performance are likely to become closer to the targets or when the prediction model is unable to make a prediction, and refraining from performing the second round when the prediction model predicts that the measurements of performance are unlikely to become closer to the targets.

An apparatus comprising:

• a non-transitory computer-readable medium storing a set of hyperparameters for an artificial intelligence (AI) model, the hyperparameters configured to be adjusted according to a hyperparameter selection technique based on one or more parameters;
• and
• a processor configured to:
• train a prediction model using a machine learning process, the prediction model configured to estimate whether further application of the hyperparameter selection technique will cause an improvement in at least one of the hyperparameters;
• select the hyperparameters using the hyperparameter selection technique;
• apply the prediction model to determine if further adjustment of the hyperparameters is likely to improve the success metric;
• and
• terminate the hyperparameter selection technique when:
• an accuracy of the prediction model in predicting improvement in at least one of the hyperparameters is above a predetermined accuracy threshold, and the prediction model predicts that further application of the hyperparameter selection technique will not result in an improvement to the hyperparameter;
• or
• the accuracy of the prediction model in predicting improvement in the hyperparameter is below the predetermined accuracy threshold, and an accuracy of hyperparameter adjustment is determined to be below a predetermined adjustment accuracy threshold.