Systems and methods for grasping objects like humans using robot grippers

A robot system, comprising:

• an object mesh module configured to determine a polygon mesh of an exterior surface of an object to be grasped by a gripper of a robot;
• a hand mesh module configured to determine a polygon mesh of an exterior surface of a human hand grasping the object;
• a gripper mesh module configured to determine a polygon mesh of an exterior surface of the gripper of the robot;
• and
• an actuation module configured to actuate actuators of the robot based on minimizing a total loss determined based on:
• an object penetration loss determined based on the polygon mesh of the exterior surface of the gripper relative to the polygon mesh of the exterior surface of the object;
• a contact heatmap loss determined based on (a) the polygon mesh of the exterior surface of the gripper relative to the polygon mesh of the exterior surface of the object and (b) the polygon mesh of the exterior surface of the human hand relative to the polygon mesh of the exterior surface of the object;
• and
• an orientation loss determined based on an orientation of vectors of the gripper relative to an orientation of vectors of the human hand.

A method, comprising:

• based on a demonstration of a human hand grasping an object, determining (a) a first vector that is normal to a palm of the human hand, (b) a second vector that is parallel to the palm of the human hand, and (c) a position of the human hand;
• determining (a) a third vector that is normal to a palm of a gripper of a robot, (b) a fourth vector that is parallel to the palm of the gripper of the robot, and (c) a present position of the gripper;
• (1) actuating one or more actuators of the robot and moving the gripper when the gripper is open such that the present position of the gripper is at the position of the human hand, the third vector is aligned with the first vector, and the fourth vector is aligned with the second vector;
• after (1), (2) actuating one or more actuators of the robot and closing fingers of the gripper based on minimizing an object penetration loss;
• and
• when the object penetration loss is minimized, (3) actuating one or more actuators of the robot and actuating the fingers of the gripper to minimize a total loss determined based on the object penetration loss and at least one other loss.

A method, comprising:

• determining a polygon mesh of an exterior surface of an object to be grasped by a gripper of a robot;
• determining a polygon mesh of an exterior surface of a human hand grasping the object;
• determining a polygon mesh of an exterior surface of the gripper of the robot;
• determining an object penetration loss based on the polygon mesh of the exterior surface of the gripper relative to the polygon mesh of the exterior surface of the object;
• determining a contact heatmap loss based on (a) the polygon mesh of the exterior surface of the gripper relative to the polygon mesh of the exterior surface of the object and (b) the polygon mesh of the exterior surface of the human hand relative to the polygon mesh of the exterior surface of the object;
• determining an orientation loss determined based on an orientation of vectors of the gripper relative to an orientation of vectors of the human hand;
• and
• actuating actuators of the robot based on minimizing a total loss determined based on the object penetration loss, the contact heatmap loss, and the orientation loss.