Real-Time Production Scheduling with Deep Reinforcement Learning and Monte Carlo Tree Research
AssigneeSiemens Aktiengesellschaft
InventorsChengtao Wen, Wei Xi Xia, Juan L. Aparicio Ojea
Life of this patent
Estimating the term…
Abstract
Systems and methods provide real-time production scheduling by integrating deep reinforcement learning and Monte Carlo tree search. A manufacturing process simulator is used to train a deep reinforcement learning agent to identify the sub-optimal policies for a production schedule. A Monte Carlo tree search agent is implemented to speed up the search for near-optimal policies of higher quality from the sub-optimal policies.
Claims20
A method for real time production scheduling, the method comprising:
- identifying a current state of a manufacturing process in a manufacturing facility;
- inputting the state into a neural network trained to generate a plurality of first scheduling policies given an input state of the production schedule;
- identifying, using a Monte Carlo tree search, one or more second scheduling policies from the plurality of first scheduling policies;
- and
- generating an updated production schedule using the one or more second scheduling policies.
A method for generating a production schedule, the method comprising:
- performing a plurality of simulations of production schedules using simulation data from a manufacturing process simulator;
- sampling actions from the plurality of simulations using domain knowledge;
- training a neural network using reinforcement learning and Monte Carlo tree search, the training identifies polices for a current state of a production schedule that lead to a positive reward;
- outputting a trained neural network for use in generating sub-optimal scheduling policies;
- optimizing output scheduling polices from the trained neural network using the Monte Carlo tree search;
- and
- generating near-optimal scheduling polices for a manufacturing process in a manufacturing facility from the optimized output scheduling policies.
A system for real time production scheduling, the system comprising:
- a production simulator configured to generate simulation data of operation of a manufacturing process over time;
- a deep reinforcement learning agent configured to input the simulation data and output one or more sub-optimal scheduling policies;
- and
- a Monte Carlo tree search agent configured to identify near optimal policies from the sub-optimal scheduling policies.
Dependencies are inferred from the claim text.