Back to results
US 11,904,467 B2Granted
System and methods for pixel based model predictive control
AssigneeGoogle Inc.
InventorDanijar Hafner
Life of this patent
Estimating the term…
Abstract
Techniques are disclosed that enable model predictive control of a robot based on a latent dynamics model and a reward function. In many implementations, the latent space can be divided into a deterministic portion and stochastic portion, allowing the model to be utilized in generating more likely robot trajectories. Additional or alternative implementations include many reward functions, where each reward function corresponds to a different robot task.
Claims20
A method implemented by one or more processors, comprising:
- training a latent robot dynamics model using unsupervised robot trajectories, wherein each of the unsupervised robot trajectories includes a corresponding sequence of:
- partial robotic observations, each of the partial robotic observations being for a corresponding time step of the sequence, and
- robotic actions, each of the robotic actions being for a corresponding time step of the sequence;
- identifying supervised robot task trajectories for a robot task, wherein each of the supervised robot task trajectories includes a corresponding sequence of:
- partial task robotic observations during a corresponding performance of the robot task,
- task robotic actions during the corresponding performance of the robot task, and
- labeled task rewards for the corresponding performance of the robot task;
- training a reward function for the robot task using the supervised robot task trajectories;
- controlling a robot to perform the robot task, wherein controlling the robot to perform the robot task comprises:
- determining a sequence of actions for the robot using both the trained robot latent dynamics model and the trained reward function for the robot task;
- and
- controlling the robot by implementing the sequence of actions.
9 more elements
A method implemented by one or more processors, comprising:
- identifying unsupervised robot trajectories, wherein each of the unsupervised robot trajectories includes:
- observation images, each of the observation images capturing a corresponding robot and being for a corresponding time step of the sequence, and
- robotic actions, each of the robotic actions being for a corresponding time step of the sequence;
- training a latent robot dynamics model using the unsupervised robot trajectories;
- using the trained latent robot dynamics model in latent planning for one or more robotic control tasks through generation, at each of a plurality of time steps and using the trained robot dynamics model, of a corresponding deterministic activation vector.
2 more elements
Dependencies are inferred from the claim text.