Execution of a genetic algorithm with variable evolutionary weights of topological parameters for neural network generation and training

#Computers are often used to solve complex quantitative and qualitative problems. For problems that involve a large data set, a specially trained professional, known as a data scientist, is often hired. The data scientist interprets the data set and constructs models that can be processed by computers to solve the problem. However, hiring data scientists is cost prohibitive for many organizations.

#For certain types of problems, advanced computing techniques, such as genetic algorithms or backpropagation, may be available to develop a model, such as a neural network, that is comparable in accuracy to a model that would be created by a data scientist. However, genetic algorithms may take a large number of iterations to converge on an acceptable neural network, and backpropagation may be slow when a large data set is being modeled or when the neural network includes a large number of nodes, connections, or layers. Additionally, backpropagation may result in “overfitting” (e.g., a neural network generated by backpropagation may model the specific data set used for training, but may not be “general” enough to model other data sets within particular tolerances).

#Furthermore, various types of machine-learning problems exist. For example, regression problems involve evaluating a series of inputs to predict a numeric output, classification problems involve evaluating a series of inputs to predict a categorical output, and reinforcement learning involves performing actions within an environment to optimize some notion of a positive reward. Due to the differences in the various types of problems, the available mechanisms to generate and train a neural network or other machine learning solution may be problem-specific. For example, a support vector machine (SVM) may be suitable for some classification problems, linear regression may be suitable for some regression problems, and a specialized machine learning platform, such as TensorFlow, may be suitable for reinforcement learning or other numerical calculations. Although a SVM, linear regression, and specialized machine learning platforms may solve specific machine-learning problems, at least some of these mechanisms may not be particularly suited to training neural networks. Thus, generating and training neural networks that meet performance requirements for each of multiple types of problems faced by an enterprise may be slow and difficult.

#FIG. 1 illustrates a particular example of a system that is operable to support cooperative execution of a genetic algorithm having variable evolutionary weights of topological parameters and a backpropagation trainer;

#FIG. 2 illustrates a particular example of a model including data representative of a neural network;

#FIG. 3 illustrates particular examples of first and second stages of operation at the system of FIG. 1;

#Referring to FIG. 1, a particular illustrative example of a system 100 is shown. The system 100, or portions thereof, may be implemented using (e.g., executed by) one or more computing devices, such as laptop computers, desktop computers, mobile devices, servers, and Internet of Things devices and other devices utilizing embedded processors and firmware or operating systems, etc. In the illustrated example, the system 100 includes a genetic algorithm 110. In some implementations, the system 100 further includes a backpropagation trainer 180. In other implementations, the system 100 does not include the backpropagation trainer 180. The backpropagation trainer 180 is an example of an optimization trainer, and other examples of optimization trainers that may be used in conjunction with the described techniques include, but are not limited to, a derivative free optimizer (DFO), an extreme learning machine (ELM), etc.

#It is to be understood that operations described herein as being performed by the genetic algorithm 110 or the backpropagation trainer 180 may be performed by a device executing the genetic algorithm 110 or the backpropagation trainer 180. In particular aspects, the genetic algorithm 110 is executed on a different device, processor (e.g., central processor unit (CPU), graphics processing unit (GPU) or other type of processor), processor core, and/or thread (e.g., hardware or software thread) than the backpropagation trainer 180. Moreover, execution of certain operations associated with the genetic algorithm 110 or the backpropagation trainer 180 may be parallelized, as further described herein.

#The genetic algorithm 110 may automatically generate a neural network model of a particular data set, such as an illustrative input data set 102. The system 100 may provide an automated model building process that enables even inexperienced users to quickly and easily build highly accurate models based on a specified data set. Additionally, the system 100 may simplify the neural network model to avoid overfitting and to reduce computing resources required to run the model. Additionally, probabilities of changing particular topological parameters of models generated during epochs of the genetic algorithm 110 may be varied from epoch to epoch based on metrics, such as fitness values, associated with models of a previous epoch. Changing the probabilities that particular topological parameters may change may improve the efficiency of the system 100 by increasing a likelihood that particular topological parameter modifications that improve fitness are performed during the model building process, thereby improving the speed with which the genetic algorithm converges and improving the fitness of models that are output by the model building process.