Back to results
US 11,693,392 B2Granted
System for manufacturing dispatching using deep reinforcement and transfer learning
AssigneeHitachi, Ltd.
InventorsShuai Zheng, Chetan Gupta, Susumu Serita
Life of this patent
Estimating the term…
Abstract
Example implementations described herein are directed to a system for manufacturing dispatching using reinforcement learning and transfer learning. The systems and methods described herein can be deployed in factories for manufacturing dispatching for reducing job-due related costs. In particular, example implementations described herein can be used to reduce massive data collection and reduce model training time, which can eventually improve dispatching efficiency and reduce factory cost.
Claims15
A method, comprising:
- generating a trained deep learning model configured to provide instructions to a factory process of a factory, the generating comprising:
- obtaining parameters from the factory process to derive a state of the factory process, the parameters comprising slack time;
- determining instructions for the factory process based on applying a deep learning model on the factory process trained against lateness and tardiness;
- providing instructions to the factory process for execution at the factory;
- obtaining state transitions of the state from updated parameters of the factory process including completion time as received from the factory;
- calculating a reward for the instructions provided to update the deep learning model, the reward based on deriving lateness and tardiness from predetermined job due and the completion time;
- and
- transferring the deep learning model from the factory to another factory to optimize dispatching process, the transferring comprising:
- generating random source states associated with the factory;
- generating random target states associate with the another factory;
- computing a projection function between the factory and the another factory based on the random source states and the random target states;
- generating source state trajectories based on optimal policy of the factory determined from the trained deep learning model;
- converting the source state trajectories into target state trajectories;
- executing a recovery process to determine the deep learning model for the another factory from the target state trajectories;
- and
- applying the deep learning model to provide instructions to the another factory,
- wherein the reward is calculated by dividing the derived lateness and tardiness over the completion time.
14 more elements
A non-transitory computer readable medium, storing instructions for executing a process, the instructions comprising:
- generating a trained deep learning model configured to provide instructions to a factory process of a factory, the generating comprising:
- obtaining parameters from the factory process to derive a state of the factory process, the parameters comprising slack time;
- determining instructions for the factory process based on applying a deep learning model on the factory process trained against lateness and tardiness;
- providing instructions to the factory process for execution at the factory;
- obtaining state transitions of the state from updated parameters of the factory process including completion time as received from the factory;
- calculating a reward for the instructions provided to update the deep learning model, the reward based on deriving lateness and tardiness from predetermined job due and the completion time;
- and
- transferring the deep learning model from the factory to another factory to optimize dispatching process, the transferring comprising:
- generating random source states associated with the factory;
- generating random target states associate with the another factory;
- computing a projection function between the factory and the another factory based on the random source states and the random target states;
- generating source state trajectories based on optimal policy of the factory determined from the trained deep learning model;
- converting the source state trajectories into target state trajectories;
- executing a recovery process to determine the deep learning model for the another factory from the target state trajectories;
- and
- applying the deep learning model to provide instructions to the another factory,
- wherein the reward is calculated by dividing the derived lateness and tardiness over the completion time.
14 more elements
An apparatus configured to control a factory process of a factory through instructions, the apparatus comprising:
- a memory configured to store a trained deep learning model;
- and
- a processor, configured to:
- generate the trained deep learning model configured to provide the instructions, by:
- obtaining parameters from the factory process to derive a state of the factory process, the parameters comprising slack time;
- determining instructions for the factory process based on applying a deep learning model on the factory process trained against lateness and tardiness;
- providing instructions to the factory process for execution at the factory;
- obtaining state transitions of the state from updated parameters of the factory process including completion time as received from the factory;
- calculating a reward for the instructions provided to update the deep learning model, the reward based on deriving lateness and tardiness from predetermined job due and the completion time;
- and
- transferring the deep learning model from the factory to another factory to optimize dispatching process, the transferring comprising:
- generating random source states associated with the factory;
- generating random target states associate with the another factory;
- computing a projection function between the factory and the another factory based on the random source states and the random target states;
- generating source state trajectories based on optimal policy of the factory determined from the trained deep learning model;
- converting the source state trajectories into target state trajectories;
- executing a recovery process to determine the deep learning model for the another factory from the target state trajectories;
- and
- applying the deep learning model to provide instructions to the another factory,
- wherein the reward is calculated by dividing the derived lateness and tardiness over the completion time.
17 more elements
Dependencies are inferred from the claim text.