Autonomous manipulation of flexible primary packaging in dimensionally stable secondary packaging by means of robots

A system for automatically manipulating flexible primary packaging in open dimensionally stable secondary packaging, together called a package, the system comprising:

• a robot module comprising a robot having at least one robot arm with a clamping gripper installed at a tool center point of each robot arm, wherein each tool center point comprises a force-torque sensor which can provide at least measurements of the force and torque direction in degrees of freedom;
• an image recording module for recording images of at least an upper segment of the primary packaging, comprising at least two stereo cameras for recording 3-D images, wherein at least one of the stereo cameras is placed to a side of an air line between the package and the robot;
• one or more processors configured to control the robot module and the image recording module to carry out the following steps:
• recording images by means of the image recording module;
• processing images and providing a three-dimensional point cloud from the received images;
• reconstructing a surface of at least the primary packaging from the three-dimensional point cloud;
• calculating a highest point of the primary packaging from the surface reconstruction of the primary packaging as a gripping point for the clamping gripper;
• calculating a movement sequence of the robot arm, comprising a collision-free approach path to the calculated gripping point for the clamping gripper;
• forwarding the calculated movement sequence to the robot module;
• receiving the measurements from the force-torque sensors;
• and
• gripping the primary packaging at the gripping point, and moving the gripper;
• wherein, in response to moving the gripper:
• the primary packaging is opened by virtue of the gripper being pulled upwards in an unscrewing manner in a movement or a rotational movement, taking into account measurements from the force-torque sensor at its tool center point, until a predefined upper limit for the stress of the primary packaging, measured by the force-torque sensor, is reached and/or predefined criteria for opening the primary packaging are achieved;
• or
• the primary packaging is closed by virtue of the gripper being moved in a twisting and/or downward manner, taking into account measurements from the force-torque sensor at its tool center point, until a predefined upper limit for the force-torque sensor is reached and/or predefined criteria for closing the primary packaging are achieved;
• and
• interfaces between the robot module and the one or more processors and between the image recording module and the one or more processors.