One-Shot Robust Visual-Force Robotic Servoing for Novel Object Insertion with 6-DoF Tracking

A system for automated assembly and insertion of objects, comprising:

• a robotic arm including links connected by joints having actuators and encoders, and a gripper of an end-effector of the robotic arm configured to grasp and release a target object in response to robot control signals;
• vision sensors configured to continuously provide visual observations of an environment;
• a memory configured to store a task demonstration image, data from the vision sensors, arm motion generation programs, a pose estimation program, and data representative of an environment, the environment including components of the automated assembly including a target object and a receptacle object;
• and
• a processor, in connection with the memory, configured to perform steps of:
• estimating, from a single task-demonstration image, an end-effector's pose and a pre-assemble object pose of the target object;
• identifying and tracking an object pose of the target object from environment images;
• generating a grasp on the target object and grasping the target object from a random position and orientation using the end-effector by determining a sequence of motions;
• moving the robotic arm to align the object pose and the pre-assemble object pose determining a sequence of motions, wherein each of the pose estimation program and the arm motion generation programs is iteratively performed for pose estimation and control of the robotic arm at each time during the moving;
• assembling the object with the receptacle object using the robotic arm by performing an impedance control search using a search pattern.