Automated model hyperparameter determination and serving

A method, comprising:

• receiving a request to determine optimal hyperparameters for a model, wherein the request comprises user defined optimization parameters comprising an input data source, a target data source, and an initial model configuration source;
• determining, using a processor, a framework extension to implement the model, wherein the framework extension comprises a model trainer for the model;
• determining an optimization assembly based at least in part on the user defined optimization parameters;
• determining a cluster to execute the optimization assembly, wherein the cluster comprises a master node and worker nodes, and wherein the model trainer is distributed to the worker nodes;
• determining an optimization script for the optimization assembly using an optimization script template and the user defined optimization parameters;
• causing the cluster to execute the optimization script to determine the hyperparameters, wherein executing the optimization script comprises:
• providing experiment parameters to the distributed model trainers;
• importing an input data from the input data source and a target data from the target data source;
• transforming the input data and the target data into transformed data in a data store format, wherein the transformed data comprises NumPy arrays allowing short-circuiting;
• persisting the transformed data in a common data store used by all of the distributed model trainers;
• determining an instance of hyperparameter values for a trial of a trained model;
• producing the trained model using the instance of the hyperparameter values;
• determining evaluation results from an evaluation of the trained model using the transformed data and short circuiting;
• determining whether there is a next trial of a next trained model;
• and
• in response to a determination that there is not the next trial of the next trained model, providing the optimal hyperparameters, wherein the optimal hyperparameters are based at least in part on the evaluation results;
• and
• building an optimal model using the optimal hyperparameters.

A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:

• receiving a request to determine optimal hyperparameters for a model, wherein the request comprises user defined optimization parameters comprising an input data source, a target data source, and an initial model configuration source;
• determining, using a processor, a framework extension to implement the model, wherein the framework extension comprises a model trainer for the model;
• determining an optimization assembly based at least in part on the user defined optimization parameters;
• determining a cluster to execute the optimization assembly, wherein the cluster comprises a master node and worker nodes, and wherein the model trainer is distributed to the worker nodes;
• determining an optimization script for the optimization assembly using an optimization script template and the user defined optimization parameters;
• causing the cluster to execute the optimization script to determine the hyperparameters, wherein executing the optimization script comprises:
• providing experiment parameters to the distributed model trainers;
• importing an input data from the input data source and a target data from the target data source;
• transforming the input data and the target data into transformed data in a data store format, wherein the transformed data comprises NumPy arrays allowing short-circuiting;
• persisting the transformed data in a common data store used by all of the distributed model trainers;
• determining an instance of hyperparameter values for a trial of a trained model;
• producing the trained model using the instance of the hyperparameter values;
• determining evaluation results from an evaluation of the trained model using the transformed data and short circuiting;
• determining whether there is a next trial of a next trained model;
• and
• in response to a determination that there is not the next trial of the next trained model, providing the optimal hyperparameters, wherein the optimal hyperparameters are based at least in part on the evaluation results;
• and
• building an optimal model using the optimal hyperparameters.