Application of simple random search approach for reinforcement learning to controller tuning parameters
AssigneeHoneywell International Inc.
InventorsNathan Lawrence, Philip D. Loewen, Bhushan Gopaluni, Gregory E. Stewart
Life of this patent
Estimating the term…
Abstract
A method and system for reinforcement learning can involve applying a finite-difference approach to a controller, and tuning the controller in response to applying the finite-difference approach by taking a state as an entirety of a closed-loop step response. The disclosed finite-different approach is based on a random search to tuning the controller, which operates on the entire closed-loop step-response of the system and iteratively improves the gains towards a desired closed-loop response. This allows for prescribing stability requirement into the reward function without any modeling procedures.
Claims20
A method for reinforcement learning to automate tuning of a controller to control a process in an industrial plant, comprising:
- supplying, by a summation unit, an output of a controller to an industrial plant in a closed loop structure;
- processing a policy update by feeding an output of the industrial plant to a total reward through a memory of the closed loop structure;
- providing a perturb policy based on an output of the policy update;
- varying parameters of the policy update to provide a finite-difference approach to tune the controller from a closed-loop step-response using the closed loop structure;
- applying the finite-difference approach to tune the controller;
- and
- tuning the controller in response to applying the finite-difference approach by taking a state as an entirety of the closed-loop step response.
A system for reinforcement learning to automate tuning of a controller to control a process in an industrial plant, comprising:
- at least one processor;
- and
- a non-transitory computer-usable medium embodying computer program code, said computer-usable medium capable of communicating with said at least one processor, said computer program code comprising instructions executable by said at least one processor and configured for:
- supplying, by a summation unit, output of a controller to a plant in a closed loop structure;
- processing a policy update by feeding an output of the plant to a total reward through a memory of the closed loop structure;
- providing a perturb policy based on the output of the policy update;
- varying parameters of the policy update to provide a finite-difference approach to tune the controller from a closed-loop step-response using the closed loop structure;
- applying the finite-difference approach to tune the controller;
- and
- tuning the controller in response to applying the finite-difference approach by taking a state as an entirety of the closed-loop step response.
An apparatus for reinforcement learning to automate tuning of a controller to control a process in an industrial plant, comprising:
- at least one processing device configured to:
- supply, by a summation unit, an output of a controller to an industrial plant in a closed loop structure
- process a policy update by feeding an output of the industrial plant to a total reward through a memory of the closed loop structure;
- provide perturb policy based on an output of the policy update;
- and
- vary parameters of the policy update to provide a finite-difference approach to tune the controller from a closed-loop step-response using the closed loop structure,
- wherein the finite-difference approach is applied to tuning the controller and the controller is tuned in response to applying the finite-difference approach by taking a state as an entirety of the closed-loop step response.
Dependencies are inferred from the claim text.