Back to results
US 12,340,307 B2Granted
Future prediction, using stochastic adversarial based sampling, for robotic control and/or other purpose(s)
AssigneeGoogle Inc.
InventorsAnthony Jacob Piergiovanni, Anelia Angelova, Alexander Toshev, Michael Ryoo
Life of this patent
Estimating the term…
Abstract
Techniques are disclosed that enable the generation of predicted sequences of terminals using a generator model portion of a prediction model. Various implementations include controlling actuators of a robot based on the predicted sequences of terminals. Additional or alternative implementations include jointly training the generator model portion of the prediction model using a discriminator model portion of the prediction model using, for example, stochastic adversarial based sampling.
Claims18
A method implemented by one or more processors, the method comprising:
- generating a predicted sequence of terminals for use in controlling one or more actuators of a robot, wherein each of the terminals represents corresponding predicted activity in an environment of the robot at a corresponding future time step, and wherein generating the predicted sequence of terminals comprises:
- receiving one or more observations, of the environment, captured using one or more sensors of the robot;
- determining an initial non-terminal by processing the one or more observations using a machine learning model, wherein the initial non-terminal is an encoded representation of observed activity in the environment captured in the one or more observations;
- for each of a plurality of iterations and until one or more conditions are satisfied:
- processing a corresponding non-terminal using the machine learning model to determine a corresponding set of rules for the corresponding non-terminal,
- wherein each rule, of the corresponding set of rules, includes a corresponding candidate terminal and a corresponding candidate next non-terminal, and
- wherein the initial non-terminal is used as the corresponding non-terminal that is processed in an initial iteration of the iterations;
- selecting a rule from the corresponding set of rules;
- assigning the corresponding candidate terminal, for the selected rule, as the next terminal in the predicted sequence of terminals, and
- assigning the corresponding candidate non-terminal, for the selected rule, as the corresponding non-terminal for use in generating the predicted sequence of terminals in an immediately subsequent iteration of the iterations;
- and
- controlling the one or more actuators of the robot based on the predicted sequence of terminals.
9 more elements
A method implemented by one or more processors, the method comprising:
- training a generator model portion of a prediction model to generate a predicted sequence of terminals for use in controlling one or more actuators of a robot,
- wherein each of the terminals in the predicted sequence is a corresponding representation of predicted activity in an environment of the robot at a corresponding future time step, and
- wherein training the generator model portion of the prediction model comprises:
- jointly training a discriminator model portion of the prediction model with the generator model portion of the prediction model using stochastic adversarial based sampling, wherein jointly training the discriminator model portion of the prediction model with the generator model portion of the prediction model comprises:
- generating a predicted sequence of training terminals using the generator model portion of the prediction model;
- processing, using the discriminator model portion of the prediction model, the generated predicted sequence of training terminals and a ground truth sequence of terminals to generate output, wherein the generated output discriminates predicted sequences of training terminals from ground truth sequences of terminals;
- and
- updating one or more weights of the prediction model based on the generated output.
5 more elements
A non-transitory computer readable storage medium storing computer program instructions that when executed by one or more processors of a computing system, cause the computing system to perform the method of:
- generating a predicted sequence of terminals for use in controlling one or more actuators of a robot, wherein each of the terminals represent corresponding predicted activity in an environment of the robot at a corresponding future time step, and wherein generating the predicted sequence of terminals comprises:
- receiving one or more observations, of the environment, captured using one or more sensors of the robot;
- determining an initial non-terminal by processing the one or more observations using a machine learning model, wherein the initial non-terminal is an encoded representation of observed activity in the environment captured in the one or more observations;
- for each of a plurality of iterations and until one or more conditions are satisfied:
- processing a corresponding non-terminal using the machine learning model to determine a corresponding set of rules for the corresponding non-terminal,
- wherein each rule, of the corresponding set of rules, includes a corresponding candidate terminal and a corresponding candidate next non-terminal, and
- wherein the initial non-terminal is used as the corresponding non-terminal that is processed in an initial iteration of the iterations;
- selecting a rule from the corresponding set of rules;
- assigning the corresponding candidate terminal, for the selected rule, as the next terminal in the predicted sequence of terminals, and
- assigning the corresponding candidate non-terminal, for the selected rule, as the corresponding non-terminal for use in generating the predicted sequence of terminals in an immediately subsequent iteration of the iterations;
- and
- controlling the one or more actuators of the robot based on the predicted sequence of terminals.
9 more elements
A computing system configured to perform the method of:
- generating a predicted sequence of terminals for use in controlling one or more actuators of a robot, wherein each of the terminals represents corresponding predicted activity in an environment of the robot at a corresponding future time step, and wherein generating the predicted sequence of terminals comprises:
- receiving one or more observations, of the environment, captured using one or more sensors of the robot;
- determining an initial non-terminal by processing the one or more observations using a machine-learning model, wherein the initial non-terminal is an encoded representation of observed activity in the environment captured in the one or more observations;
- for each of a plurality of iterations and until one or more conditions are satisfied:
- processing a corresponding non-terminal using the machine learning model to determine a corresponding set of rules for the corresponding non-terminal,
- wherein each rule, of the corresponding set of rules, includes a corresponding candidate terminal and a corresponding candidate next non-terminal, and
- wherein the initial non-terminal is used as the corresponding non-terminal that is processed in an initial iteration of the iterations;
- selecting a rule from the corresponding set of rules;
- assigning the corresponding candidate terminal, for the selected rule, as the next terminal in the predicted sequence of terminals, and
- assigning the corresponding candidate non-terminal, for the selected rule, as the corresponding non-terminal for use in generating the predicted sequence of terminals in an immediately subsequent iteration of the iterations;
- and
- controlling the one or more actuators of the robot based on the predicted sequence of terminals.
9 more elements
A non-transitory computer-readable storage medium storing instructions executable by one or more processors of a computing system to perform the method of:
- generating a predicted sequence of terminals for use in controlling one or more actuators of a robot, wherein each of the terminals in the predicted sequence represents corresponding predicted activity in an environment of the robot at a corresponding future time step, and wherein generating the predicted sequence of terminals comprises:
- receiving one or more observations, of the environment, captured using one or more sensors of the robot;
- determining an initial non-terminal by processing the one or more observations using a machine-learning model, wherein the initial non-terminal is an encoded representation of observed activity in the environment captured in the one or more observations;
- for each of a plurality of iterations and until one or more conditions are satisfied:
- processing a corresponding non-terminal using the machine learning model to determine a corresponding set of rules for the corresponding non-terminal,
- wherein each rule, of the corresponding set of rules, includes a corresponding candidate terminal and a corresponding candidate next non-terminal, and
- wherein the initial non-terminal is used as the corresponding non-terminal that is processed in an initial iteration of the iterations;
- selecting a rule from the corresponding set of rules;
- assigning the corresponding candidate terminal, for the selected rule, as the next terminal in the predicted sequence of terminals, and
- assigning the corresponding candidate non-terminal, for the selected rule, as the corresponding non-terminal for use in generating the predicted sequence of terminals in an immediately subsequent iteration of the iterations;
- and
- controlling the one or more actuators of the robot based on the predicted sequence of terminals.
9 more elements
Dependencies are inferred from the claim text.