Simultaneous hyper parameter and feature selection optimization using evolutionary boosting machines

A machine learning system, comprising:

• one or more processors;
• and
• memory storing instructions that, when executed by the one or more processors, cause the machine learning system to optimize hyper parameter and feature selection by causing the machine learning system to:
• identify a training data source comprising a plurality of records, wherein each record of the plurality of records comprises data corresponding to a plurality of features;
• initialize a first cycle by generating a first generation of candidate models, wherein generating each respective first candidate model of the first generation of candidate models comprises selecting a first subset of features, of the plurality of features, for use in the first candidate model;
• determine, for each first candidate model of the first generation of candidate models, a respective first optimal boosting iterations hyper parameter;
• evaluate fitness values for each respective first candidate model in the first generation based on a corresponding subset of features and a corresponding optimal boosting iterations hyper parameter;
• perform the first cycle by iteratively evaluating a plurality of generations of the machine learning system, wherein evaluating a respective generation of the machine learning system comprises:
• generating a second generation of candidate models by performing an evolution process on a respective subset of features associated with selected models of a current generation of candidate models to generate respective second candidate models;
• determining, for each second candidate model of the second generation of candidate models, a respective second optimal boosting iterations hyper parameter;
• and
• evaluating fitness values for each respective second candidate model in the second generation based on a corresponding subset of features and a corresponding optimal boosting iterations hyper parameter;
• subsequent to iteratively evaluating the plurality of generations, determine a selected candidate model of a final generation of candidate models of the first cycle;
• determine that the selected candidate model breaches a solution constraint, wherein the solution constraint defines a threshold number of selected features and a target optimal boosting iterations range, and the breach is determined when evaluation of the selected candidate model produces a solution that does not satisfy the solution constraint;
• adjust a learning rate of the machine learning system and a subset of features in response to determining that the selected candidate model breaches the solution constraint;
• perform one or more second cycles employing the adjusted learning rate and the adjusted subset of features, wherein a first generation of at least one second cycle of the one or more second cycles is generated based on the final generation of the first cycle;
• and
• based on termination criteria, identify a resulting candidate model of a final cycle of the one or more second cycles as an optimized model.

One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause a machine learning system to perform steps comprising:

• identifying a training data source comprising a plurality of records, wherein each record of the plurality of records comprises data corresponding to a plurality of features;
• initializing a first cycle of the machine learning system by generating a first generation of candidate models, wherein generating each respective first candidate model of the first generation of candidate models comprises selecting a first subset of features, of the plurality of features, for use in the first candidate model;
• determining, for each first candidate model of the first generation of candidate models, a respective first optimal boosting iterations hyper parameter;
• evaluating fitness values for each respective first candidate model in the first generation based on a corresponding subset of features and a corresponding optimal boosting iterations hyper parameter;
• performing one or more cycles of the machine learning system by iteratively evaluating a plurality of generations of the machine learning system, wherein evaluating a respective generation of the machine learning system comprises:
• generating a second generation of candidate models by performing an evolution process on a respective subset of features associated with selected models of a current generation of candidate models to generate respective second candidate models;
• determining, for each second candidate model of the second generation of candidate models, a respective second optimal boosting iterations hyper parameter;
• and
• evaluating fitness values for each respective second candidate model in the second generation based on a corresponding subset of features and a corresponding optimal boosting iterations hyper parameter;
• subsequent to iteratively evaluating the plurality of generations, determining a selected candidate model of a final generation of candidate models associated with the one or more cycles;
• determining that the selected candidate model breaches a solution constraint, wherein the solution constraint defines a threshold number of selected features and a target optimal boosting iterations range, and the breach is determined when evaluation of the selected candidate model produces a solution that does not satisfy the solution constraint;
• adjusting a learning rate of the machine learning system and a subset of features in response to determining that the selected candidate model breaches the solution constraint;
• performing one or more additional cycles of the machine learning system employing the adjusted learning rate and the adjusted subset of features;
• and
• based on termination criteria, identifying a resulting candidate model of a final cycle of the machine learning system as an optimized model.