System and Method For Loss Function Metalearning For Faster, More Accurate Training, and Smaller Datasets

A computer-readable medium storing instructions that, when executed by a computer, perform a process for partially training a neural network to discover and optimize a candidate loss function comprising:

• initiating a random population of candidate loss functions, wherein each candidate loss function includes a set of optimizable coefficients;
• (i) training the random population of candidate loss functions on training data;
• (ii) evaluating a fitness of each candidate loss function based on a performance thereof on the training data in accordance with a fitness function;
• (iii) selecting one or more select candidate loss functions in accordance with the fitness evaluations;
• (iv) reproducing using the select candidate loss functions in accordance with reproduction processes to establish and discover a next population of candidate loss functions;
• (v) repeating steps (i) to (iv) for nth generations;
• selecting one or more best candidate loss functions in accordance with the fitness evaluations from an nth generation population of candidate functions;
• optimizing the set of coefficients of each of the one or more best candidate loss functions using an optimization process wherein the optimization process is a covariance-matrix adaptation evolutionary strategy (CMA-ES);
• and
• implementing an optimized best candidate loss function to train one or more predictive models.

An automated machine learning process comprising:

• evolving candidate loss functions by a genetic algorithm running on at least one processor and selecting one or more best candidate loss functions in accordance with a fitness evaluation, wherein the candidate loss function is a tree having multiple nodes;
• optimizing by an optimization process running on at least one processor a set of coefficients of each of the one or more best candidate loss functions, wherein the set of coefficients is represented by a vector with dimensionality equal to the number of nodes in the best candidate loss function's tree and the optimization process is a covariance-matrix adaptation evolutionary strategy (CMA-ES);
• and
• implementing an optimized best candidate loss function to train one or more predictive models.

An automated machine learning system comprising:

• a first subsystem including at least one processor programmed to perform a process of evolving candidate loss functions and selecting one or more best candidate loss functions in accordance with a fitness evaluation, wherein the candidate loss function is a tree having multiple nodes and the programmed process for evolving candidate loss functions is a genetic algorithm;
• a second subsystem including at least one processor programmed to perform a process of optimizing a set of coefficients of each of the one or more best candidate loss functions, wherein the set of coefficients is represented by a vector with dimensionality equal to the number of nodes in the best candidate loss function's tree and the programmed process optimizing a set of coefficients of each of the one or more best candidate loss functions is a covariance-matrix adaptation evolutionary strategy (CMA-ES);
• and
• a third subsystem including at least one processor programmed to perform a process of implementing an optimized best candidate loss function to train one or more predictive models.