Precise Placement Based on On-the-Fly Object Pose Estimation

A robotic system, comprises:

• a robot comprising a plurality of joints, wherein the plurality of joints comprise a first joint that rotates the entire robot along a fixed base and one or more other joints;
• an image capturing device mounted onto a link connected to the first joint, wherein the image capturing device is configured to move along with the first joint of the robot and obtain one or more images of an object held by the robot;
• and
• a control system configured to:
• move the robot from a first workspace to a second workspace based on controlling the first joint to rotate the robot along the fixed base, wherein the robot holds the object and the object corresponds to a first tool center point (TCP);
• obtain, using the image capturing device, one or more images of the object while the robot is moving from the first workspace to the second workspace;
• determine a second TCP based on the one or more images;
• control the one or more other joints of the robot to adjust the object held by the robot according to the second TCP;
• and
• control the robot to place the object at a target location of the second workspace according to a refined TCP based on the adjustment result converging to a predetermined reference.