Self-learning industrial robotic system
AssigneeHitachi, Ltd.
InventorsHeming Chen, Yi-Chu Chang
Life of this patent
Estimating the term…
Abstract
Example implementations described herein are directed to a simulation environment for a real world system involving one or more robots and one or more sensors. Scenarios are loaded into a simulation environment having one or more virtual robots corresponding to the one or more robots, and one or more virtual sensors corresponding to the one or more virtual system to train a control strategy model from reinforcement learning, which is subsequently deployed to the real world environment. In cases of failure of the real world environment, the failures are provided to the simulation environment to generate an updated control strategy model for the real world environment.
Claims14
A method, comprising:
- loading a scenario into a simulation environment comprising one or more virtual robots and one or more virtual sensors;
- executing a control strategy model on the simulation environment to operate the one or more virtual robots based on the scenario;
- and
- training the control strategy model from feedback from the one or more virtual sensors of the simulation environment,
- wherein the control strategy model is configured to assign an expected reward for each operation completed by the one or more virtual robots in the simulation environment, wherein the training the control strategy model from the feedback from the one or more virtual sensors of the simulation environment comprises:
- for the each operation completed by the one or more virtual robots in the simulation environment:
- determining a reward to be assigned for the each operation completed by the one or more virtual robots in the simulation environment based on sensor data from the one or more virtual sensors;
- and
- updating the control strategy model based on the determined reward to be assigned and the expected reward to be assigned by the control strategy model.
A method for a control system of a real world environment comprising one or more robots and one or more sensors, the method comprising:
- executing a control strategy model trained by a simulation environment comprising one or more virtual robots corresponding to the one or more robots and one or more virtual sensors corresponding to the one or more sensors, the control strategy model configured to control the one or more robots based on sensor data of the one or more sensors,
- monitoring the real world environment for one or more failure scenarios detected from sensor data of the one or more sensors;
- and
- providing the one or more failure scenarios to the simulation environment,
- wherein the control strategy model is configured to assign an expected reward for each operation completed by the one or more virtual robots in the simulation environment, wherein training of the control strategy model comprises:
- for the each operation completed by the one or more virtual robots in the simulation environment:
- determining a reward to be assigned for the each operation completed by the one or more virtual robots in the simulation environment based on sensor data from the one or more virtual sensors;
- and
- updating the control strategy model based on the determined reward to be assigned and the expected reward to be assigned by the control strategy model.
A first server communicatively coupled to a second server configured to manage a real world environment comprising one or more robots and one or more sensors, the first server configured to manage a simulation environment comprising one or more virtual robots corresponding to the one or more robots, and one or more virtual sensors corresponding to the one or more sensors, the first server comprising:
- a processor, configured to:
- load a scenario into the simulation environment;
- execute a control strategy model on the simulation environment to operate the one or more virtual robots based on the scenario;
- and
- train the control strategy model from feedback from the one or more virtual sensors of the simulation environment,
- wherein the control strategy model is configured to assign an expected reward for each operation completed by the one or more virtual robots in the simulation environment, wherein the processor is configured to train the control strategy model from the feedback from the one or more virtual sensors of the simulation environment by:
- for the each operation completed by the one or more virtual robots in the simulation environment:
- determining a reward to be assigned for the each operation completed by the one or more virtual robots in the simulation environment based on sensor data from the one or more virtual sensors;
- and
- updating the control strategy model based on the determined reward to be assigned and the expected reward to be assigned by the control strategy model.
Dependencies are inferred from the claim text.