Back to results
US 12,619,213 B2Granted
Controlling method and device for an industrial device
AssigneeSiemens Aktiengesellschaft
InventorsXiang Li, Xiao Feng Wang, Fan Bo Meng
Life of this patent
Estimating the term…
Abstract
Various embodiments include methods for controlling an industrial device. Some embodiments include: obtaining a state input characterizing a current state of the industrial device; processing the state input to generate an action output characterizing an expected action to be performed by the industrial device for the current state, based on a machine learning model trained based on states of the industrial device, actions each performed for each state of the industrial device and results each obtained by performing each action; and generating a control signal for the industrial device based on the action output.
Claims12
A method for controlling an industrial device, the method comprising:
- obtaining a state input characterizing a current state of the industrial device;
- processing the state input to generate an action output characterizing an expected action to be performed by the industrial device for the current state, based on a machine learning model trained based on states of the industrial device, respective actions performed for each state of the industrial device, and results respective obtained by performing each action;
- generating a control signal for the industrial device based on the action output;
- and
- updating the machine learning model by:
- obtaining a further state of the industrial device;
- processing the further state of the industrial device to generate a further action to be performed by the industrial device for the further state, based on the machine learning model;
- generating a state next to the further state based on the further state and the further action based on a device model that is trained based on (a) one or more state-action pairs, each comprising a respective state of the industrial device and a corresponding action performed for the respective state and (b) one or more next states each corresponding to a respective state-action pair;
- computing a result of the further action based on the further state and the state next to the further state;
- and
- updating the machine learning model based on the further state, the further action, and the result of the further action.
8 more elements
A device for controlling an industrial device, the device comprising:
- an obtaining apparatus for obtaining a state input characterizing a current state of the industrial device;
- a processor for processing the state input to generate an action output characterizing an expected action to be performed by the industrial device for the current state, based on a machine learning model that is trained based on states of the industrial device, actions each performed for each state of the industrial device and results each obtained by performing each action, and generating a control signal for the industrial device based on the action output;
- and
- a controller for controlling the industrial device based on the control signal;
- and
- wherein the processor is further programmed to update the machine learning model by:
- obtaining a further state of the industrial device;
- processing the further state of the industrial device to generate a further action to be performed by the industrial device for the further state, based on the machine learning model;
- generating a state next to the further state based on the further state and the further action based on a device model that is trained based on (a) one or more state-action pairs, each comprising a respective state of the industrial device and a corresponding action performed for the respective state and (b) one or more next states each corresponding to a respective state-action pair;
- computing a result of the further action based on the further state and the state next to the further state;
- and
- updating the machine learning model based on the further state, the further action, and the result of the further action.
9 more elements
A controlling device for an industrial device, the controlling device comprising:
- one or more processor;
- and
- one or more memories storing instructions operable, when executed the one or more processor, to cause the one or more processor to:
- obtain a state input characterizing a current state of the industrial device;
- process the state input to generate an action output characterizing an expected action to be performed by the industrial device for the current state, based on a machine learning model trained based on states of the industrial device, actions each performed for each state of the industrial device and results each obtained by performing each action;
- generate a control signal for the industrial device based on the action output;
- and
- update the machine learning model by:
- obtaining a further state of the industrial device;
- processing the further state of the industrial device to generate a further action to be performed by the industrial device for the further state, based on the machine learning model;
- generating a state next to the further state based on the further state and the further action based on a device model that is trained based on (a) one or more state-action pairs, each comprising a respective state of the industrial device and a corresponding action performed for the respective state and (b) one or more next states each corresponding to a respective state-action pair;
- computing a result of the further action based on the further state and the state next to the further state;
- and
- updating the machine learning model based on the further state, the further action, and the result of the further action.
11 more elements
Dependencies are inferred from the claim text.