Systems and methods for training a reinforcement learning system for pallet routing in a manufacturing environment

A system for training a reinforcement learning system, the system comprising:

• a processor;
• and
• a nontransitory computer-readable medium including instructions that are executable by the processor, wherein the instructions include:
• obtaining state information from one or more sensors of a digital twin, wherein the state information includes a number of a plurality of pallets at a first routing control location of the digital twin during a manufacturing simulation, a type of the plurality of pallets at the first routing control location during the manufacturing simulation, or a combination thereof;
• determining an action at the first routing control location based on the state information, wherein the action includes one of a pallet merging operation and a pallet splitting operation;
• determining a consequence state based on the action, wherein the consequence state includes a consequent number of the plurality of pallets at a consequent routing control location of the digital twin, a type of the plurality of pallets at the consequent routing control location, or a combination thereof;
• calculating a transient production value based on the consequence state and a transient objective function;
• calculating a steady state production value based on the consequence state and a steady state objective function;
• and
• selectively adjusting one or more reinforcement parameters of the reinforcement learning system based on the transient production value and the steady state production value, wherein the transient objective function and the steady state objective function are based on a first target production value associated with a first type of pallets from among the plurality of pallets and a second target production value associated with a second type of pallets from among the plurality of pallets.

A method for training a reinforcement learning system, the method comprising:

• obtaining state information from one or more sensors of a digital twin, wherein the state information includes a number of a plurality of pallets at a first routing control location of the digital twin during a manufacturing simulation;
• determining an action at the first routing control location based on the state information, wherein the action includes one of a pallet merging operation and a pallet splitting operation;
• determining a consequence state based on the action, wherein the consequence state includes a consequent number of the plurality of pallets at a consequent routing control location of the digital twin, a type of the plurality of pallets at the consequent routing control location, or a combination thereof;
• calculating a transient production value based on the consequence state and a transient objective function;
• calculating a steady state production value based on the consequence state and a steady state objective function;
• and
• selectively adjusting one or more reinforcement parameters of the reinforcement learning system based on the transient production value and the steady state production value, wherein the one or more reinforcement parameters include a sensor layout of the one or more sensors, the transient objective function, the steady state objective function, or a combination thereof, and wherein the transient objective function and the steady state objective function are based on a first target production value associated with a first type of pallets from among the plurality of pallets and a second target production value associated with a second type of pallets from among the plurality of pallets.