Techniques for adaptive robotic assembly

One or more non-transitory computer-readable storage media including instructions that, when executed by at least one processor, cause the at least one processor to perform steps for controlling a robotic system, the steps comprising:

• receiving sensor data associated with one or more parts;
• executing, based on the sensor data, a first trained machine learning model that predicts one or more grasp proposals associated with the one or more parts, wherein each grasp proposal indicates a region of one of the one or more parts that the robotic system can grasp, wherein the robotic system comprises a first robot and a second robot;
• causing the first robot to grasp a first part in a first pose based on the one or more grasp proposals, wherein the first part is included in the one or more parts;
• receiving additional sensor data associated with the first part;
• executing, based on the additional sensor data, a second trained machine learning model that predicts a pose proposal associated with a second pose of the first part;
• in response to determining that the grasping of the first part in the first pose does not allow the first part to be assembled with the one or more parts, computing one or more movements of the robotic system to grasp the first part in the second pose by the first robot or the second robot, wherein the second pose of the first part is different than the first pose of the first part;
• determining one or more additional movements of the robotic system to assemble the first part with the one or more parts;
• and
• causing the robotic system to perform the one or more movements and the one or more additional movements.

A robotic system comprising:

• one or more memories storing instructions;
• and
• one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to:
• receive sensor data associated with one or more parts,
• execute, based on the sensor data, a first trained machine learning model that predicts one or more grasp proposals associated with the one or more parts, wherein each grasp proposal indicates a region of one of the one or more parts that the robotic system can grasp, wherein the robotic system comprises a first robot and a second robot;
• cause the first robot to grasp a first part in a first pose based on the one or more grasp proposals, wherein the first part is included in the one or more parts;
• receive additional sensor data associated with the first part;
• execute, based on the additional sensor data, a second trained machine learning model that predicts a pose proposal associated with a second pose of the first part;
• in response to determining that the grasping of the first part in the first pose does not allow the first part to be assembled with the one or more parts, compute one or more movements of the robotic system to grasp the first part in the second pose by the first robot or the second robot, wherein the second pose of the first part is different than the first pose of the first part;
• compute one or more additional movements of the robotic system to assemble the first part with the one or more parts;
• and
• cause the robotic system to perform the one or more movements and the one or more additional movements.