Method and electronic device for selecting deep neural network hyperparameters

An electronic device adapted for analyzing a plurality of hyperparameter configurations of a target neural network model, wherein each of the hyperparameter configurations comprises a plurality of hyperparameters, the electronic device comprising:

• a storage device, storing one or more commands;
• and
• a processor, coupled to the storage device, the processor being configured to execute the commands to:
• sample a plurality of testing hyperparameter configurations from hyperparameter ranges of the plurality of hyperparameters;
• train the target neural network model by using a first training dataset and the plurality of testing hyperparameter configurations, and obtain a plurality of first accuracies corresponding to the plurality of testing hyperparameter configurations after training for first preset epochs;
• perform a hyperparameter recommendation operation, wherein the hyperparameter recommendation operation comprises:
• obtaining an interpolation function based on an observation point set and a recommended list of hyperparameter configurations by using a model-based optimization algorithm, wherein the observation point set comprises the plurality of testing hyperparameter configurations and the plurality of first accuracies, and the recommended list of hyperparameter configurations comprises a plurality of recommended hyperparameter configurations;
• determining a pre-trained model respectively corresponding to each of the plurality of recommended hyperparameter configurations by using a trainable-weight-selecting method;
• respectively training the pre-trained models corresponding with the plurality of recommended hyperparameter configurations by using a second training dataset and the plurality of recommended hyperparameter configurations, and obtaining a plurality of second accuracies corresponding to the plurality of recommended hyperparameter configurations after training for second preset epochs;
• and
• predicting a plurality of final accuracies corresponding to the plurality of second accuracies by using a predictive early stopping model;
• and
• select a first recommended hyperparameter configuration corresponding to the final accuracy having a highest predicted value as a hyperparameter setting for continuing training the target neural network model.