A system for performing a model search and providing a system output based on the model search, the system comprising a processor and a memory, the system configured to:
- store, in the memory, a plurality of features extracted from a training dataset;
- store, in the memory, one or more objectives and one or more pre-hoc constraints;
- with the processor, generate a search space and store the search space in the memory, wherein generating the search space comprises mapping a plurality of feature costs to the extracted features, sampling the search space based on the one or more objectives and the one or more pre-hoc constraints, and generating and storing one or more seed candidates from the sampled search space, the one or more seed candidates comprising one or more models each having one or more weights;
- with the processor, iteratively optimize the one or more seed candidates until an exit condition is met by iteratively performing one or more iterations in a loop, each iteration comprising steps of:
- applying the set of pre-hoc constraints to the seed candidates to quantify a fitness level of each of the seed candidates;
- evaluating the fitness level of each of the seed candidates by comparing the fitness level to the one or more objectives and the one or more pre-hoc constraints;
- storing a plurality of candidates from the plurality of seed candidates based on the one or more objectives and the one or more pre-hoc constraints;
- training the candidates by applying one or more predetermined training techniques, wherein the one or more predetermined training techniques comprise adjusting the one or more weights of each of the candidates;
- following the training of the candidates, evaluating the candidates based on post-hoc constraints, wherein the post-hoc constraints comprise at least one of performance, robustness, and explainability;
- forming or updating, based on a fitness level of one or more candidates, a Pareto front comprising the one or more candidates, wherein the Pareto front is formed based on a plurality of dimensions and wherein the one or more candidates are fitted to the Pareto front based on the evaluating of the one or more candidates following training;
- checking if at least one of the one or more exit conditions has been reached, wherein
- if the exit conditions have not been reached, regenerating seed candidates identified based on the stored plurality of candidates on the Pareto front and performing another iteration of optimizing the seed candidates, or
- if at least one of the exit conditions has been reached, selecting one or more final candidates based on the seed candidates in the Pareto front, and exiting the loop;
- and
- storing, in the memory, the one or more final candidates, and performing the system output based on the one or more final candidates.