Method, System, and Computer Program Product for Implementing Reinforcement Learning

A system for implementing reinforcement learning by a neural network, comprising:

• at least one processor;
• and
• at least one non-transitory computer readable medium comprising instructions to direct the at least one processor to:
• for each epoch of a first predetermined number of epochs, perform a second predetermined number of training iterations and a third predetermined number of testing iterations using a first neural network, the first neural network comprising a first set of parameters, the training iterations comprising a first set of hyperparameters, and the testing iterations comprising a second set of hyperparameters;
• divide the testing iterations into segments, each segment comprising a fourth predetermined number of testing iterations;
• determine a first pattern based on at least one of the segments;
• and
• adjust at least one of the first set of hyperparameters or the second set of hyperparameters based on the pattern,
• wherein, for each training iteration, the instructions further direct the at least one processor to:
• receive first state data indicating a first state of an environment;
• determine a first action from a set of possible actions based on the first state data using the first neural network;
• perform the first action within the environment;
• receive second state data indicating a second state of the environment after performance of the first action and first reward data indicating a first reward achieved in the environment by performance of the first action;
• and
• store first action data indicating the first action, the first reward data, the first state data, and the second state data,
• wherein, for a subset of the training iterations, the instructions further direct the at least one processor to update the first set of parameters of the first neural network based on at least one of the action data, the reward data, the first state data, and the second state data, wherein the subset of the training iterations is determined based on the first set of hyperparameters,
• wherein the instructions further direct the at least one processor to copy the first set of parameters from the first neural network to a second neural network after a fourth predetermined number of training iterations, and
• wherein updating the first set of parameters comprises:
• determining a first predicted reward based on the first state data using the first neural network;
• determining a second predicted reward based on the second state data and the first reward data using the second neural network;
• and
• adjusting the first set of parameters to reduce a difference between the first predicted reward and the second predicted reward.

A computer program product for implementing reinforcement learning by a neural network, the computer program product comprising at least one non-transitory computer-readable medium including one or more instructions that, when executed by at least one processor, cause the at least one processor to:

• for each epoch of a first predetermined number of epochs, perform a second predetermined number of training iterations and a third predetermined number of testing iterations using a first neural network, the first neural network comprising a first set of parameters, the training iterations comprising a first set of hyperparameters, and the testing iterations comprising a second set of hyperparameters;
• divide the testing iterations into segments, each segment comprising a fourth predetermined number of testing iterations;
• determine a first pattern based on at least one of the segments;
• and
• adjust at least one of the first set of hyperparameters or the second set of hyperparameters based on the pattern,
• wherein, for each training iteration, the instructions further direct the at least one processor to:
• receive first state data indicating a first state of an environment;
• determine a first action from a set of possible actions based on the first state data using the first neural network;
• perform the first action within the environment;
• receive second state data indicating a second state of the environment after performance of the first action and first reward data indicating a first reward achieved in the environment by performance of the first action;
• and
• store first action data indicating the first action, the first reward data, the first state data, and the second state data,
• wherein, for a subset of the training iterations, the instructions further direct the at least one processor to, update the first set of parameters of the first neural network based on at least one of the action data, the reward data, the first state data, and the second state data, wherein the subset of the training iterations is determined based on the first set of hyperparameters,
• wherein the instructions further direct the at least one processor to copy the first set of parameters from the first neural network to a second neural network after a fourth predetermined number of training iterations, and
• wherein updating the first set of parameters comprises determining a first predicted reward based on the first state data using the first neural network, determining a second predicted reward based on the second state data and the first reward data using the second neural network, and adjusting the first set of parameters to reduce a difference between the first predicted reward and the second predicted reward.