System and method for parametric system evaluation

An automated evaluation system, comprising:

• an input/output interface that receives a parameter space comprising one or more parameters corresponding to an application and one or more search functions for selecting a parameter or a parameter combination from the parameter space for evaluating a characteristic corresponding to the system;
• an iteration controller stored in memory communicatively coupled to the input/ output interface, wherein the iteration controller iteratively selects a parameter or parameter combination from the parameter space using the one or more search functions;
• and
• an iteration processor communicatively coupled to the iteration controller, wherein the iteration processor:
• reconfigures source code corresponding to the application to generate reconfigured source code before compilation of the source code into executable code so as to receive the selected parameter or the selected parameter combination and the one or more search functions;
• invokes a creation command for preparing the application for execution using the selected parameter or the selected parameter combination, wherein invoking the creation command comprises creating a binary representation of the reconfigured source code;
• invokes an execution command that controls execution of the binary representation of the reconfigured source code by selectively configuring one or more parameterized compiler flags, command line arguments, environment variables, data files to be read in by the application at run time, or combinations thereof, and executes the application using the selected parameter or the selected parameter combination;
• monitors the execution of the application to determine metrics associated with the desired system characteristic;
• configures the iteration controller to update the one or more search functions for iteratively selecting the parameter or the parameter combination from the parameter space for executing the application that improves the desired characteristic of the system based on the determined metrics associated with the desired system characteristic;
• directs the iteration controller to iteratively select another parameter or another parameter combination from the parameter space using the one or more updated search functions;
• and
• executes the application using the selected another parameter or the selected another parameter combination;
• wherein the application is programmed to receive the selected parameters, each of the selected parameters turned on or turned off by preprocessor definitions at compile time or through conditional execution at runtime such that the preprocessor definitions feedback into a preprocessor prior to compilation of the reconfigured source code.