A non-transitory computer-readable storage medium on which are stored instructions that, when executed, cause a processor of a programmable device to perform operations of:
- receiving first problem information from a first computing device of a first user, the first problem information identifying a first objective function representing a first black-box optimization problem for which a solution is being requested from one or more second users, and input information indicating one or more inputs to the first objective function;
- causing the first problem information to be presented on a first problem user interface of a virtual exchange, the virtual exchange reducing one or more of computing, network, and memory resource for optimizing objective functions associated with black-box problems by providing a collaborative environment in which users submit candidate solutions for optimizing a respective objective function associated with a respective black-box problem, wherein the first problem user interface includes:
- a description of the first problem,
- a first candidate solution queue comprising candidate solutions to the optimization problem that have been provided by one or more second users,
- a list of processed candidate solutions including candidate solutions that have been obtained from the first candidate solution queue and provided as an input to a first simulator to execute the first objective function with a respective one of the candidate solutions as the input to the first objective function, and wherein
- the user interface is accessible to the first user from the first computing device and one or more second computing devices wherein the one or more second computing devices are associated with a respective one of the one or more second users, and wherein all entries of the first candidate solution queue and all entries of the list of processed candidate solutions are visible to all registered users accessing the first problem user interface;
- receiving a first candidate solution to the first objective function from a respective computing device of the one or more second computing devices associated with a respective user of the one or more second users, wherein the first candidate solution includes one or more first candidate input values to be provided as an input to the objective function;
- inserting the first candidate solution into the candidate solution queue associated with the first problem;
- storing information associated with the first candidate solution in a candidate solution information data structure stored in a memory of the data processing system, wherein storing the information in the candidate solution information data structure includes:
- determining that the first candidate solution exceeds a size of a candidate solution field included in the candidate solution data structure allocated for storing the first candidate solution,
- storing the first candidate solution in a memory location external to the candidate information data structure, and
- inserting a link to the memory location in the candidate solution field of the candidate solution information data structure;
- automatically causing the first problem user interface to be updated to display the first candidate solution in the first candidate solution queue;
- receiving, over a network connection, a candidate solution request for one or more candidate solutions for the first black-box optimization problem from a remote simulator service configured to implement the first simulator to securely execute the first objective function on candidate solutions without disclosing the first objective function to the one or more second users;
- providing, over the network connection, a candidate solution message including the first candidate solution as the input to a first simulator in response to the candidate solution request;
- obtaining, as an output from the first simulator of the simulator service, a first objective value generated by the first objective function responsive to executing the first objective function on the first candidate solution;
- removing the first candidate solution from the first candidate solution queue;
- inserting the first candidate solution and the first objective value into a list of processed candidate solutions;
- automatically causing the first problem user interface to be updated to display the first candidate solution and the first objective value in the list of processed candidate solutions;
- determining a first improvement value provided by the first candidate solution by analyzing one or more previously determined solutions to the first black-box optimization problem and the first objective value to determine an amount of improvement provided by the first candidate solution;
- determining a first reward by analyzing the first improvement value using a reward function associated with the first problem;
- notifying the second user that first candidate solution has earned the second user the first reward;
- and
- automatically updating the first problem user interface to display the first reward with the first candidate solution and the first objective value.