Back to results
US 12,569,984 B2Granted
System and methods for pixel based model predictive control
AssigneeGoogle Inc.
InventorDanijar Hafner
Life of this patent
Estimating the term…
Abstract
Techniques are disclosed that enable model predictive control of a robot based on a latent dynamics model and a reward function. In many implementations, the latent space can be divided into a deterministic portion and stochastic portion, allowing the model to be utilized in generating more likely robot trajectories. Additional or alternative implementations include many reward functions, where each reward function corresponds to a different robot task.
Claims16
A robot comprising:
- one or more actuators;
- memory storing a trained latent robot dynamics model and a trained reward function for a robot task;
- one or more processors configured to:
- predict multiple candidate sequences of actions using a latent state observation of the robot and the latent robot dynamics model;
- select, from the multiple candidate sequences of actions, using the trained reward function, a sequence of actions for performing the robot task;
- and
- control one or more actuators based on the selected sequence of actions to cause performance of the robot task.
4 more elements
A robot comprising:
- one or more actuators;
- memory storing a trained latent robot dynamics model and a trained reward function for a robot task;
- one or more processors configured to:
- use a latent state observation of the robot, the latent robot dynamics model, and the trained reward function, to determine a sequence of actions for performing the robot task;
- and
- control one or more actuators based on the determined sequence of actions to cause performance of the robot task;
- wherein, subsequent to controlling the one or more actuators based on the determined sequence of actions to cause performance of the robot task, one or more of the processors are further configured to:
- identify an updated latent state observation of the robot;
- use the updated latent state observation of the robot, the latent robot dynamics model, and the trained reward function, to determine an updated sequence of actions for performing the robot task;
- and
- control the one or more actuators based on the determined updated sequence of actions to cause performance of the robot task.
8 more elements
A robot comprising:
- one or more actuators;
- memory storing a trained latent robot dynamics model and a trained reward function for a robot task;
- one or more processors configured to:
- use a latent state observation of the robot, the latent robot dynamics model, and the trained reward function, to determine a sequence of actions for performing the robot task;
- and
- control one or more actuators based on the determined sequence of actions to cause performance of the robot task;
- wherein, subsequent to controlling the one or more actuators based on the determined sequence of actions to cause performance of the robot task, one or more of the processors are further configured to:
- use an additional latent state observation of the robot, the latent robot dynamics model, and the trained reward function, to determine an additional sequence of actions for performing an additional robot task;
- and
- control the one or more actuators based on the additional sequence of actions to cause performance of the additional robot task.
7 more elements
A method implemented by one or more processors, the method comprising:
- predicting multiple candidate sequences of actions using a latent state observation of a robot and a latent robot dynamics model;
- selecting, from the multiple candidate sequences of actions, using a trained reward function a sequence of actions for performing a robot task, wherein the trained reward function is for the robot task;
- and
- controlling one or more actuators based on the selected sequence of actions to cause performance of the robot task.
1 more element
A method implemented by one or more processors, the method comprising:
- using a latent state observation of a robot, a latent robot dynamics model, and a trained reward function, to determine a sequence of actions for performing a robot task, wherein the trained reward function is for the robot task;
- and
- controlling one or more actuators based on the determined sequence of actions to cause performance of the robot task;
- wherein, subsequent to controlling the one or more actuators based on the determined sequence of actions to cause performance of the robot task, the method further comprises:
- identifying an updated latent state observation of the robot;
- using the updated latent state observation of the robot, the latent robot dynamics model, and the trained reward function, to determine an updated sequence of actions for performing the robot task;
- and
- controlling the one or more actuators based on the determined updated sequence of actions to cause performance of the robot task.
5 more elements
A method implemented by one or more processors, the method comprising:
- using a latent state observation of a robot, a latent robot dynamics model, and a trained reward function, to determine a sequence of actions for performing a robot task, wherein the trained reward function is for the robot task;
- and
- controlling one or more actuators based on the determined sequence of actions to cause performance of the robot task;
- wherein, subsequent to controlling the one or more actuators based on the determined sequence of actions to cause performance of the robot task, the method further comprises:
- using an additional latent state observation of the robot, the latent robot dynamics model, and the trained reward function, to determine an additional sequence of actions for performing an additional robot task;
- and
- controlling the one or more actuators based on the additional sequence of actions to cause performance of the additional robot task.
4 more elements
Dependencies are inferred from the claim text.