Universal approximation of reinforcement learning solutions with hypernetworks
AssigneeSamsung Electronics Co., Ltd.
InventorsSahand Rezaei-Shoshtari, David Meger, Francois Robert Hogan, Gregory Lewis Dudek, Charlotte Morissette
Life of this patent
Estimating the term…
Abstract
Provided is a method for training a hypernetwork to provide a policy for use on a previously-unseen task. The hypernetwork may be trained at a robot factory and then shipped with a robot. At the point of deployment, the robot may be given a context for the previously-unseen task. The robot then uses the context and the hypernetwork to create a policy for performing the previously-unseen task. The policy represents an artificial intelligence machine generated for the previously-unseen task.
Claims20
A method of training a hypernetwork, the method comprising:
- initializing the hypernetwork;
- sampling a mini-batch of system parameter sets from a plurality of system parameter sets;
- generating, using the hypernetwork, policy weights for a policy;
- generating, using the hypernetwork, value function weights for a value function;
- calculating a first loss, L_pred, using the mini-batch;
- calculating a second loss, L_TD, using the mini-batch;
- updating the hypernetwork using the first loss and the second loss;
- and
- repeating the sampling through the updating until the hypernetwork has converged.
An apparatus comprising:
- one or more processors;
- and
- one or more memories, the one or more memories storing instructions configured to cause the apparatus to at least:
- initialize a hypernetwork;
- sample a mini-batch of system parameter sets from a plurality of system parameter sets;
- generate, using the hypernetwork, policy weights for a policy;
- generate, using the hypernetwork, value function weights for a value function;
- calculate a first loss, L_pred, using the mini-batch;
- calculate a second loss, L_TD, using the mini-batch;
- update the hypernetwork using the first loss and the second loss;
- and
- repeatedly perform the sample through update operations until the hypernetwork has converged.
A non-transitory computer readable medium storing instructions, the instructions configured to cause an apparatus to at least:
- initialize a hypernetwork;
- sample a mini-batch of system parameter sets from a plurality of system parameter sets;
- generate, using the hypernetwork, policy weights for a policy;
- generate, using the hypernetwork, value function weights for a value function;
- calculate a first loss, L_pred, using the mini-batch;
- calculate a second loss, L_TD, using the mini-batch;
- update the hypernetwork using the first loss and the second loss;
- and
- repeatedly perform the sample through update operations until the hypernetwork has converged.
Dependencies are inferred from the claim text.