Bound enhanced reinforcement learning system for distribution supply chain management

A method for training a reinforcement learning algorithm comprising:

• initializing a bifurcated memory structure including a first replay buffer configured to store environmental transition data, and a second replay buffer configured to store demand data, the bifurcated memory structure being structured to reduce a quantity of data subject to reinforcement learning analysis by enabling selective sampling from different data types during bound calculation;
• initializing a first state-action value function network having first random weights, and a second state-action value function network having second random weights;
• determining an action based on an exploration policy;
• executing the action in a simulated environment and storing transition results in the bifurcated memory structure;
• calculating an upper bound and a lower bound of optimal costs based on the the stored transition results;
• performing a penalty-constrained gradient descent on the first state-action value function network using a combination of a loss function and penalty terms that penalize state-action values violating the calculated bounds, the penalty-constrained gradient descent implementing a processing architecture to modify neural network weight updates by applying the penalty terms to reduce a computational training requirement compared to conventional neural network algorithms;
• and
• deploying the trained reinforcement learning algorithm for real-time control operations, wherein the trained reinforcement learning algorithm operates using the bifurcated memory structure to provide control decisions in real-time.

A system comprising:

• initialize a bifurcated memory structure including a first replay buffer configured to store environmental transition data, and a second replay buffer configured to store demand data, the bifurcated memory structure being structured to reduce a quantity of data subject to reinforcement learning analysis by enabling selective sampling from different data types during bound calculation;
• initializing a first state-action value function network having first random weights, and a second state-action value function network having second random weights;
• determining an action based on an exploration policy;
• executing the action in a simulated environment and storing transition results in the bifurcated memory structure;
• calculating an upper bound and a lower bound of optimal costs based on the the stored transition results;
• perform a penalty-constrained gradient descent on the first state-action value function network using a combination of a loss function and penalty terms that penalize state-action values violating the calculated bounds, the penalty-constrained gradient descent implementing a processing architecture to modify neural network weight updates by applying the penalty terms to reduce a computational training requirement compared to conventional neural network algorithms;
• and
• deploy the trained reinforcement learning algorithm for real-time control operations, wherein the trained reinforcement learning algorithm operates using the bifurcated memory structure to provide control decisions in real-time.