Determining final grasp pose of robot end effector after traversing to pre-grasp pose

A method implemented by one or more processors of a robot, the method comprising:

• generating, based on first vision data from a first vision component of the robot, a pre-grasp pose and an initial grasp pose for grasping of an object that is in an environment of the robot;
• providing, to actuators of the robot, commands that direct an end effector of the robot to traverse to the pre-grasp pose;
• subsequent to providing the commands that direct the end effector of the robot to traverse to the pre-grasp pose, and prior to attempting a grasp of the object using the end effector:
• capturing one or more instances of second vision data using a second vision component of the robot,
• wherein the second vision component of the robot is coupled to the end effector, and
• wherein translation of the end effector causes corresponding translation of the second vision component;
• generating current visual features based on at least one of the instances of the second vision data,
• generating a candidate grasp pose by applying one or more pre-stored grasp pose criteria to the current visual features;
• determining that the candidate grasp pose satisfies a distance threshold relative to the initial grasp pose;
• in response to determining that the candidate grasp pose satisfies the distance threshold relative to the initial grasp pose, determining a final grasp pose, for grasping of the object, based at least in part on the candidate grasp pose, and
• calculating a path to move the end effector to the final grasp pose;
• and
• after determining the final grasp pose:
• providing, to the actuators of the robot, commands that cause the end effector of the robot to traverse the path, and attempt a grasp of the object.

A robot comprising:

• an end effector;
• actuators;
• a first vision component coupled to a head or a body of the robot;
• a second vision component coupled to the end effector and translating with translation of the end effector;
• one or more processors that execute stored instructions to:
• generate, based on first vision data from the first vision component, a pre-grasp pose for grasping of an object that is in an environment of the robot;
• provide, to the actuators, commands that direct the end effector to traverse to the pre-grasp pose;
• subsequent to providing the commands that direct the end effector of the robot to traverse to the pre-grasp pose, and prior to attempting a grasp of the object using the end effector:
• capture one or more instances of second vision data using the second vision component;
• generating current visual features based on at least one of the instances of the second vision data,
• select, from a plurality of discrete pre-stored visual features, at least given pre-stored visual features based on the given pre-stored visual features satisfying one or more conditions relative to the current visual features,
• determine a candidate grasp pose using one or more grasp pose criteria based on the grasp pose criteria being stored in association with the selected given pre-stored visual features,
• determine a final grasp pose, for grasping of the object, based at least in part on the candidate grasp pose, and
• calculate a path to move the end effector to the final grasp pose;
• and
• after determining the final grasp pose, and in response to determining the path is kinematically feasible:
• provide, to the actuators of the robot, commands that cause the end effector of the robot to traverse the path, and attempt a grasp of the object.