Back to results
US 12,498,677 B2Granted
Mitigating reality gap through training a simulation-to-real model using a vision-based robot task model
AssigneeGoogle Inc.
InventorsKanishka Rao, Chris Harris, Julian Ibarz, Alexander Irpan, Seyed Mohammad Khansari Zadeh, Sergey Levine
Life of this patent
Estimating the term…
Abstract
Implementations disclosed herein relate to mitigating the reality gap through training a simulation-to-real machine learning model (“Sim2Real” model) using a vision-based robot task machine learning model. The vision-based robot task machine learning model can be, for example, a reinforcement learning (“RL”) neural network model (RL-network), such as an RL-network that represents a Q-function.
Claims19
A method implemented by one or more processors, the method comprising:
- processing a simulated image, using a simulation-to-real generator model, to generate a simulated episode predicted real image,
- wherein the simulated image is generated by a robotic simulator during a simulated episode of a simulated robot attempting performance of a robotic task;
- processing the simulated episode predicted real image, using a real-to-simulation generator model, to generate a simulated episode predicted simulation image;
- processing the simulated image along with a simulated robot action, using a task machine learning model being trained for use in the robotic task, to generate a first predicted value;
- processing the simulated episode predicted real image along with the simulated robot action, using the task machine learning model, to generate a second predicted value;
- processing the simulated episode predicted simulated image along with the simulated robot action, using the task machine learning model, to generate a third predicted value;
- generating a loss as a function of comparisons of the first predicted value, the second predicted value, and the third predicted value;
- and
- updating the simulation-to-real generator model based on the generated loss.
6 more elements
A system comprising:
- at least one processor;
- and
- memory storing instructions that, when executed by the at least one processor, cause the at least one processor to be operable to:
- process a simulated image, using a simulation-to-real generator model, to generate a simulated episode predicted real image,
- wherein the simulated image is generated by a robotic simulator during a simulated episode of a simulated robot attempting performance of a robotic task;
- process the simulated episode predicted real image, using a real-to-simulation generator model, to generate a simulated episode predicted simulation image;
- process the simulated image along with a simulated robot action, using a task machine learning model being trained for use in the robotic task, to generate a first predicted value;
- process the simulated episode predicted real image along with the simulated robot action, using the task machine learning model, to generate a second predicted value;
- process the simulated episode predicted simulated image along with the simulated robot action, using the task machine learning model, to generate a third predicted value;
- generate a loss as a function of comparisons of the first predicted value, the second predicted value, and the third predicted value;
- and
- update the simulation-to-real generator model based on the generated loss.
9 more elements
A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations, the operations comprising:
- processing a simulated image, using a simulation-to-real generator model, to generate a simulated episode predicted real image,
- wherein the simulated image is generated by a robotic simulator during a simulated episode of a simulated robot attempting performance of a robotic task;
- processing the simulated episode predicted real image, using a real-to-simulation generator model, to generate a simulated episode predicted simulation image;
- processing the simulated image along with a simulated robot action, using a task machine learning model being trained for use in the robotic task, to generate a first predicted value;
- processing the simulated episode predicted real image along with the simulated robot action, using the task machine learning model, to generate a second predicted value;
- processing the simulated episode predicted simulated image along with the simulated robot action, using the task machine learning model, to generate a third predicted value;
- generating a loss as a function of comparisons of the first predicted value, the second predicted value, and the third predicted value;
- and
- updating the simulation-to-real generator model based on the generated loss.
6 more elements
Dependencies are inferred from the claim text.