Policy improvement method, recording medium, and policy improvement apparatus
AssigneeFujitsu Limited
InventorTomotake Sasaki
Life of this patent
Estimating the term…
Abstract
A policy improvement method of improving a policy of reinforcement learning by a state value function, is executed by a computer and includes adding a plurality of perturbations to a plurality of components of a first parameter of the policy; estimating a gradient function of the state value function with respect to the first parameter, based on a result of an input determination performed for a control target in the reinforcement learning, the input determination being performed by using the policy that uses a second parameter obtained by adding the plurality of perturbations to the plurality of components; and updating the first parameter based on the estimated gradient function.
Claims10
A policy improvement method of improving a policy of reinforcement learning by a state value function, the policy improvement method executed by a computer and comprising:
- adding, by the computer, a plurality of perturbations to a plurality of components of a first parameter of the policy;
- estimating, by the computer, a gradient function of the state value function with respect to the first parameter, based on a result of an input determination performed for a control target in the reinforcement learning, the input determination being performed by using the policy that uses a second parameter obtained by adding the plurality of perturbations to the plurality of components;
- and
- updating, by the computer, the first parameter based on the estimated gradient function, wherein
- the perturbations respectively added to the plurality of components are determined independently of each other and are determined so that probabilities of a perturbation having a positive value or a negative value are equal where an absolute value of the positive value and an absolute value of the negative value are equal.
A non-transitory computer-readable medium storing therein a policy improvement program that improves a policy of reinforcement learning by a state value function and causes a computer to execute a process, the process comprising:
- adding, by the computer, a plurality of perturbations to a plurality of components of a first parameter of the policy;
- estimating, by the computer, a gradient function of the state value function with respect to the first parameter, based on a result of an input determination performed for a control target in the reinforcement learning, the input determination being performed by using the policy that uses a second parameter obtained by adding the plurality of perturbations to the plurality of components;
- and
- updating, by the computer, the first parameter based on the estimated gradient function, wherein
- the perturbations respectively added to the plurality of components are determined independently of each other and are determined so that probabilities of a perturbation having a positive value or a negative value are equal where an absolute value of the positive value and an absolute value of the negative value are equal.
A policy improvement apparatus that improves a policy of reinforcement learning by a state value function, the policy improvement apparatus comprising:
- a memory;
- and
- a processor coupled to the memory, the processor configured to:
- add a plurality of perturbations to a plurality of components of a first parameter of the policy;
- estimate a gradient function of the state value function with respect to the first parameter, based on a result of an input determination performed for a control target in the reinforcement learning, the input determination being performed by using the policy that uses a second parameter obtained by adding the plurality of perturbations to the plurality of components;
- and
- updating the first parameter based on the estimated gradient function, wherein
- the perturbations respectively added to the plurality of components are determined independently of each other and are determined so that probabilities of a perturbation having a positive value or a negative value are equal where an absolute value of the positive value and an absolute value of the negative value are equal.
Dependencies are inferred from the claim text.