Eye-on-hand reinforcement learner for dynamic grasping with active pose estimation

A system for performing dynamic grasping of a target object using visual sensory inputs, comprising:

• a robotic arm configured to include links connected by joints having actuators and encoders, and a gripper of an end-effector of the robotic arm configured to grasp the target object in response to robot control signals;
• sensors arranged on the robotic arm, wherein the sensors are configured to measure state measurements of the robotic arm and the target object, wherein the sensors include a vision sensor configured to continuously provide visual observations for tracking poses of the target object in a workspace and compute grasp poses;
• and
• a controller comprising:
• a data input/output interface configured to receive the state measurements of the robotic arm and the target object from the sensors;
• a memory configured to store an Eye-on-Hand (EoH) reinforcement leaner (EARL) policy, a physics-based simulator, an arm motion generation program;
• and
• a processor, in connection with the memory, configured to perform steps of:
• training the Eye-on-Hand reinforcement learner (EARL) policy, wherein
• for the training of the EARL policy, the processor is configured to train a neural network to learn a control policy for the dynamic grasping, and
• the training of the neural network to learn the control policy is based on a curriculum design that gradually increases task difficulty and adapts reward function design;
• tracking the poses of the target object moving in the workspace;
• computing a set of grasp poses on the target object and dynamically selecting a desired grasp pose on the target object moving in the workspace;
• computing robotic arm motion commands using the trained Eye-on-Hand reinforcement learner policy;
• generating the robot control signals based on the computed robotic arm motion commands;
• and
• transmitting, via the data input/output interface, the robot control signals to the actuators of the joints and the gripper to follow the target object while keeping the target object in a field of view of the vision sensor and grasp the target object in the workspace.