Training actor-critic algorithms in laboratory settings
AssigneesSony Corporation, Sony Corporation of America
InventorsPiyush Khandelwal, James MacGlashan, Peter Wurman
Life of this patent
Estimating the term…
Abstract
Reinforcement learning methods can use actor-critic networks where (1) additional laboratory-only state information is used to train a policy that much act without this additional laboratory-only information in a production setting; and (2) complex resource-demanding policies are distilled into a less-demanding policy that can be more easily run at production with limited computational resources. The production actor network can be optimized using a frozen version of a large critic network, previously trained with a large actor network. Aspects of these methods can leverage actor-critic methods in which the critic network models the action value function, as opposed to the state value function.
Claims18
A method of training an agent, comprising:
- training a policy using an actor network and a critic network using state information available in a both a laboratory setting and in a production setting, wherein the state information available in the laboratory setting is greater than the state information available in the production setting;
- optimizing the critic network with an actor network;
- and
- providing a production actor network for the agent in the production setting.
A method of training an agent, comprising:
- training a policy using an actor network and a critic network;
- optimizing the critic network with an actor network;
- providing a production actor network for the agent in a production setting;
- duplicating the critic network, when the training is complete, into a frozen critic network;
- and
- optimizing the production actor network using the frozen critic network, wherein the production actor network is smaller than the actor network.
A non-transitory computer-readable storage medium with an executable program stored thereon, wherein the program instructs one or more processors to perform the following steps:
- training a policy using an actor network and a critic network using state information available in a both a laboratory setting and in a production setting, wherein the state information available in the laboratory setting is greater than the state information available in the production setting;
- optimizing the critic network with an actor network;
- and
- providing a production actor network for the agent in the production setting.
Dependencies are inferred from the claim text.