Training neural networks for policy adaptation

A system configured to train a neural network to determine solutions to an optimization problem, the system comprising at least one processor, and computer-readable storage comprising computer-executable instructions which, when executed by the at least one processor, cause the system to:

• obtain a neural network comprising a plurality of network parameter values defining a policy, wherein the neural network is configured to obtain a state signal representing a state of an instance of the optimization problem and to process state parameters included therein according to the plurality of network parameter values to generate action selection data for selecting an action to be performed by an agent in response to the state signal;
• obtain training data representing a plurality of instances of an optimization problem, each instance of the optimization problem being represented by a set of state parameters;
• and
• for each instance of the optimization problem in the training data:
• generate a plurality of solutions for the said instance of the optimization problem, wherein each of the plurality of solutions is generated by:
• conditioning the policy of the neural network according to an N-dimensional vector;
• and
• processing a set of state parameters representing the said instance using the neural network conditioned on the N-dimensional vector whereby to apply the conditioned policy to the said instance of the optimization problem;
• evaluate each of the plurality of solutions according to a reward function to identify a said N-dimensional vector associated with a highest performing solution;
• and
• train the neural network conditioned on the said N-dimensional vector for the said instance of the optimization problem.

A computer-readable non-transitory storage medium on which is stored computer-executable instructions which, when executed by at least one processor, cause the at least one processor to:

• obtain a neural network comprising a plurality of network parameter values defining a policy, wherein the neural network is configured to obtain a state signal representing a state of an instance of the optimization problem and to process state parameters included therein according to the plurality of network parameter values to generate action selection data for selecting an action to be performed by an agent in response to the state signal;
• obtain training data representing a plurality of instances of an optimization problem, each instance of the optimization problem being represented by a set of state parameters;
• and
• for each instance of the optimization problem in the training data:
• generate a plurality of solutions for the said instance of the optimization problem, wherein each of the plurality of solutions is generated by:
• conditioning the policy of the neural network according to an N-dimensional vector;
• and
• processing a set of state parameters representing the said instance using the neural network conditioned on the N-dimensional vector whereby to apply the conditioned policy to the said instance of the optimization problem;
• evaluate each of the plurality of solutions according to a reward function to identify a said N-dimensional vector associated with a highest performing solution;
• and
• train the neural network conditioned on the said N-dimensional vector for the said instance of the optimization problem.

A method of using a neural network to generate solutions to an optimization problem, the method comprising:

• obtaining input data representing an instance of an optimization problem, the input data including a set of state parameters;
• obtaining a neural network comprising a plurality of network parameter values defining a policy, wherein the neural network has been trained to be conditioned on an N-dimensional vector selected from a vector latent space;
• determining the vector latent space;
• instructing an agent to implement a search process to identify a desired solution to the instance of the optimization problem, the search process comprising iteratively:
• selecting an N-dimensional vector;
• conditioning the policy of the neural network using the selected N-dimensional vector;
• processing the set of state parameters representing the instance using the neural network conditioned on the N-dimensional vector whereby to apply the conditioned policy to the said instance of the optimization problem to generate a solution;
• generating a set of solution state parameters representing a solution to an instance of the optimization problem;
• evaluating the solution according to a reward function;
• and
• determining an updated N-dimensional vector to be selected based on the evaluation.

A system configured to implement a neural network to determine solutions to an optimization problem, the system comprising at least one processor, and computer-readable storage comprising computer-executable instructions which, when executed by the at least one processor, cause the system to:

• obtain input data representing an instance of an optimization problem, the input data including a set of state parameters;
• obtain a neural network comprising a plurality of network parameter values defining a policy, wherein the neural network has been trained to be conditioned on an N-dimensional vector selected from a vector latent space w;
• determine the vector latent space;
• instruct an agent to implement a search process to identify a desired solution to the instance of the optimization problem, the search process comprising iteratively:
• selecting an N-dimensional vector;
• conditioning the policy of the neural network using the selected N-dimensional vector;
• processing the set of state parameters representing the instance using the neural network conditioned on the N-dimensional vector whereby to apply the conditioned policy to the said instance of the optimization problem to generate a solution;
• generating a set of solution state parameters representing a solution to an instance of the optimization problem;
• evaluating the solution according to a reward function;
• and
• determining an updated N-dimensional vector to be selected based on the evaluation.

A computer-readable non-transitory storage medium on which is stored computer-executable instructions which, when executed by at least one processor, cause the at least one processor to:

• obtain input data representing an instance of an optimization problem, the input data including a set of state parameters;
• obtain a neural network comprising a plurality of network parameter values defining a policy, wherein the neural network has been trained to be conditioned on an N-dimensional vector selected from a vector latent space w;
• determine the vector latent space;
• instruct an agent to implement a search process to identify a desired solution to the instance of the optimization problem, the search process comprising iteratively:
• selecting an N-dimensional vector;
• conditioning the policy of the neural network using the selected N-dimensional vector;
• processing the set of state parameters representing the instance using the neural network conditioned on the N-dimensional vector whereby to apply the conditioned policy to the said instance of the optimization problem to generate a solution;
• generating a set of solution state parameters representing a solution to an instance of the optimization problem;
• evaluating the solution according to a reward function;
• and
• determining an updated N-dimensional vector to be selected based on the evaluation.