Multiobjective coevolution of deep neural network architectures

An automated machine learning (AutoML) implementation system for evolving deep neural network (DNNs) to perform a predetermined task, comprising:

• an algorithm layer for evolving deep neural network (DNN) hyperparameters and architectures, the algorithm including instructions for initializing a first population DNNs each DNN comprised of modules and a blueprint containing a combination of one or more existing species of modules, wherein for each existing species of modules in the first population and during each generation in a plurality of generations of subsequent populations:
• creating a first set of empty species of modules;
• for each existing species of modules, determining Pareto front of DNNs for each existing species in accordance with a fitness value and a novelty score;
• removing DNNs in the Pareto front of each existing species and adding them to the first set of empty species to form one or more sets of new species;
• replacing each existing species with the one more sets of new species;
• truncating the one or more sets of new species by preserving a predetermined fraction of modules within each set of new species of modules, wherein the modules within each set of new species of modules are ranked in accordance with fitness value and novelty score and the predetermined fraction of modules is preserved from the modules with the highest rankings;
• generating new DNNs using procreation, including at least one of crossover and mutation;
• adding new DNNs to the first set of new species;
• a system layer for parallel training of multiple evolved DNNs received from the algorithm layer, determination of one or more fitness values and one or more novelty scores for each of the received DNNs, and providing the one or more fitness values and one or more novelty scores back to the algorithm layer for use in additional generations of evolving DNNs;
• and
• a program layer for informing the algorithm layer and system layer of one or more desired optimization characteristics of the evolved DNNs.