Black-box optimization using neural networks

One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to implement a system for determining an optimized setting for one or more process parameters, the system comprising:

• a first neural network that corresponds to a plurality of worker computing units, the first neural network having a plurality of network parameters and being configured to:
• receive a sequence of network inputs, each network input in the sequence comprising (i) a respective input setting that specifies the one or more process parameters, the one or more process parameters comprising one or more architecture settings of a machine learning model being trained using a machine learning training process, and (ii) a measure of a performance of a machine learning training process with the input setting, and
• process the sequence of network inputs in accordance with the network parameters to generate a respective network output for each network input that defines an updated setting for the one or more process parameters, and
• provide the updated setting to one of the plurality of worker computing units to evaluate performance of the machine learning training process in training the machine learning model with the updated setting;
• wherein the first neural network has been trained on a loss function that, for each training iteration of one or more training iterations, depends on values of a training function for a plurality of queries, each of the plurality of queries being defined by a respective network output generated by first neural network at a respective time step of a plurality of time steps of the training iteration;
• a subsystem configured to:
• determine, for each of a plurality of candidate settings for the one or more process parameters, a respective measure of the performance of the machine learning training process with the candidate setting, wherein the determining comprises, for each time step of a plurality of time steps, performing the following:
• providing, as input to the first neural network, a current network input comprising (i) a current setting of the one or more process parameters of the machine learning training process and (ii) a measure of performance of the machine learning training process in training the machine learning model with the current setting;
• processing, using the first neural network in accordance with first values of the network parameters, the current network input to obtain a current network output that defines an updated setting of the one or more process parameters,
• obtaining a measure of the performance of the machine learning training process in training the machine learning model with the updated setting defined by the current network output generated by the first neural network, and
• generating a new network input to be provided as input to the first neural network at a next time step, the new network input comprising (i) the updated setting defined by the current network output and (ii) the measure of the performance of the machine learning training process with the updated setting defined by the current network output;
• and
• select a candidate setting from the plurality of candidate settings as an optimized setting for the one or more process parameters using the measures of the performance for the candidate settings;
• and
• the plurality of worker computing units, wherein each worker computing unit is configured to:
• receive, from the first neural network that corresponds to the plurality of worker computing units, a respective input setting that specifies one or more process parameters of the machine learning training process, wherein the one or more process parameters are not learned as part of the machine learning training process;
• execute the machine learning training process with the one or more process parameters specified by the input setting;
• and
• measure the performance of the machine learning training process with the one or more process parameters specified by the input setting,
• wherein obtaining a measure of the performance of the machine learning training process with the updated setting defined by the current network output comprises:
• providing the updated setting defined by the current network output generated by the first neural network to one of the plurality of worker computing units and, obtaining, from the one of the plurality of worker computing units, the measure of the performance of the machine learning training process.

A method of determining an optimized setting for one or more process parameters, the method comprising:

• determining, for each of a plurality of candidate settings for the one or more process parameters, a respective measure of the performance of the machine learning training process with the candidate setting, wherein the one or more process parameters comprising one or more architecture settings of a machine learning model being trained using a machine learning training process, wherein the determining comprises, for each time step of a plurality of time steps, performing the following:
• providing, as input to a first neural network that corresponds to a plurality of worker computing units, a current network input comprising (i) a current setting of the one or more process parameters of the machine learning training process and (ii) a measure of performance of the machine learning training process in training the machine learning model with the current setting;
• processing, using the first neural network in accordance with first values of the network parameters, the current network input to obtain a current network output that defines an updated setting of the one or more process parameters, and
• provide the updated setting to one of the plurality of worker computing units to evaluate performance of the machine learning training process in training the machine learning model with the updated setting,
• obtaining a measure of the performance of the machine learning training process in training the machine learning model with an updated setting defined by the current network output generated by the first neural network, comprising providing the updated setting defined by the current network output generated by the first neural network to one of a plurality of worker computing units and, obtaining, from the one of the plurality of worker computing units, the measure of the performance of the machine learning training process,
• wherein the first neural network has been trained on a loss function that, for each training iteration of one or more training iterations, depends on values of a training function for a plurality of queries, each of the plurality of queries being defined by a respective network output generated by first neural network at a respective time step of a plurality of time steps of the training iteration,
• wherein each worker computing unit is configured to:
• receive, from the first neural network that corresponds to the plurality of worker computing units, a respective input setting that specifies one or more process parameters of the machine learning training process, wherein the one or more process parameters are not learned as part of the machine learning training process;
• execute the machine learning training process with one or more process parameters specified by the input setting;
• and
• measure the performance of the machine learning training process with the one or more process parameters specified by the input setting,
• generating a new network input to be provided as input to the first neural network at a next time step, the new network input comprising (i) the updated setting defined by the current network output and (ii) the measure of the performance of the machine learning training process with the updated setting defined by the current network output;
• and
• selecting a candidate setting from the plurality of candidate settings as an optimized setting for the one or more process parameters using the measures of the performance for the candidate settings.