Feature selection and hyperparameter optimization using lds

A computer program product for simultaneous feature selection and hyperparameter optimization of non-linear models of machine learning, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:

• set a first solution having first hyperparameters and a first set of features selected from a plurality of features of a training data set;
• initialize a weight table providing a score for each feature of the first set of features;
• initialize a discrepancy;
• perform a limited discrepancy search (LDS), according to an order based on the weight table, to obtain a second solution having second hyperparameters and a second set of features from the plurality of features by swapping the first set of features and switching the first hyperparameters from the first solution with the discrepancy, while updating the weight table during LDS;
• compare the second solution with the first solution;
• and
• obtain a new solution with improved features and improved hyperparameters, as an optimized solution.

A system for simultaneous feature selection and hyperparameter optimization of non-linear models of machine learning, the system comprising:

• a memory;
• and
• one or more processors in communication with the memory configured to:
• set a first solution having first hyperparameters and a first set of features selected from a plurality of features of a training data set;
• initialize a weight table providing a score for each feature of the first set of features;
• initialize a discrepancy;
• perform a limited discrepancy search (LDS), according to an order based on the weight table, to obtain a second solution having second hyperparameters and a second set of features from the plurality of features by swapping the first set of features and switching the first hyperparameters from the first solution with the discrepancy, while updating the weight table during LDS;
• compare the second solution with the first solution;
• and
• obtain a new solution with improved features and improved hyperparameters, as an optimized solution.