Back to results
US 2022/0101111 A1Published application
Mitigating delusional bias in deep q-learning for robotic and/or other agents
AssigneeGoogle Inc.
InventorsTyler Lu, Boon Teik Ooi, Craig Boutilier, Dale Schuurmans, DiJia Su
Life of this patent
Estimating the term…
Abstract
Delusional bias can occur in function approximation Q-learning. Techniques for training and/or using a value network to mitigate delusional bias is disclosed herein, where the value network can be used to generate action(s) for an agent (e.g., a robot agent, a software agent, etc.). In various implementations, delusional bias can be mitigated by using a soft-consistency penalty. Additionally or alternatively, delusional bias can be mitigated by using a search framework over multiple Q-functions.
Claims19
A method implemented by one or more processors, the method comprising:
- training, using Q-learning, a value network to mitigate delusional bias, wherein training the value network to mitigate delusional bias comprises one or both of:
- (1) integrating, into the Q-learning used in training the value network, a soft-consistency penalty used to bias Q-labels, across training batches, towards being consistent with an expressible policy;
- and
- (2) generating the value network using a search framework, wherein generating the value network using the search framework comprises:
- (a) generating, using the search framework, a plurality of candidate Q-regressors,
- (b) selecting a Q-regressor from the plurality of candidate Q-regressors, wherein selecting the Q-regressor is based on determining that the Q-regressor satisfies one or more conditions;
- and
- providing the trained value network for deployment on a robotic and/or other agent.
5 more elements
Dependencies are inferred from the claim text.