Transparent object bin picking

A method for picking up a transparent object from a group of transparent objects using a robot, said method comprising:

• obtaining a 2D red-green-blue (RGB) color image of the objects using a 3D camera;
• obtaining a 2D depth map image of the objects using the 3D camera, where pixels in the depth map image are assigned a value identifying the distance from the camera to the objects;
• generating a segmentation image of the objects by performing an image segmentation process using a deep learning mask R-CNN (convolutional neural network) that extracts features from the RGB image and assigns a label to pixels in the segmentation image so that each object in the segmentation image has the same label;
• and
• identifying a location for picking up the object using the segmentation image and the depth map image, wherein obtaining a color image, obtaining a depth map image, generating a segmentation image and identifying a location for picking up the object are performed each time an object is picked up from the group of objects by the robot.

A robot system for picking up an object from a group of objects using a robot, said system comprising:

• a 3D camera that provides a 2D red-green-blue (RGB) color image and a 2D depth map image of the objects;
• a deep learning mask R-CNN (convolutional neural network) that generates a segmentation image of the objects by performing an image segmentation process that extracts features from the RGB image and assigns a label to pixels in the segmentation image so that each object in the segmentation image has the same label;
• and
• means for identifying a location for picking up the object using the segmentation image and the depth map image, wherein obtaining a color image, obtaining a depth map image, generating a segmentation image and identify a location for picking up the object are performed each time an object is picked up from the group of objects by the robot.