Online augmentation of learned grasping

A method for online augmentation for learned grasping, comprising:

• identifying an action from a discrete action space for an environment of an agent, wherein the discrete action space includes a first set of grasps;
• wherein the agent is able to grasp an object in the environment, and wherein the action is a grasp that is defined as at least one contact point pair having an agent contact point associated with the agent and an object contact point associated with the object;
• identifying a second set of grasps of the agent based on a transition model and at least one contact parameter, wherein the agent is able to grasp the object in the environment, wherein a grasp is defined as at least one contact point pair having an agent contact point associated with the agent and an object contact point associated with the object, and wherein the at least one contact parameter defines allowed states of contact for the agent;
• applying a reward function to evaluate each grasp of the second set of grasps based on a set of contact forces within a friction cone that minimizes a difference between an actual net wrench on the object and a predetermined net wrench, wherein the reward function is iteratively optimized online using a lookahead tree having a plurality of nodes, each node of the plurality of nodes predicting one or more next grasp states amongst the second set of grasps;
• selecting a next grasp from the second set of grasps based on a reward value of the reward function associated with the plurality of nodes;
• and
• causing the agent to execute the next grasp.

A method for augmentation for learned grasping of an agent, comprising:

• performing object path planning to determine partial state information, wherein the object path planning includes an initial action, a next action, and an Nth action for the agent with respect to an object, wherein the initial action, the next action, and the Nth action are grasps based on at least one contact parameter, wherein a grasp is defined as at least one contact point pair having an agent contact point associated with the agent and an object contact point associated with the object, and wherein the at least one contact parameter defines allowed states of contact for the agent;
• performing object trajectory optimization by assigning timestamps to the initial action, the next action, and the Nth action;
• optimizing a reward function based on an iterative inverse kinematic procedure and wrench error to calculate a reward, wherein the reward may be based on a difference between an estimated required wrench and an actual wrench, and wherein the reward is calculated for a node at an offline time during training and an online time during operation or a test;
• generating an optimized sequence of actions based on the optimized reward function based on a deep reinforcement learning (DRL) policy trained;
• iteratively augmenting the learned grasp with an online lookahead tree having a plurality of nodes, each node of the plurality of nodes predicting one or more next grasp states, wherein the online lookahead tree generates an optimal policy between a current grasp and n-steps into the future based on the reward of the reward function associated with the plurality of nodes;
• and
• implementing the reference object trajectory including the augmented grasps via the agent.