Method and device for annotating images of an object captured using a camera

A device for annotating images of an object, comprising:

• a robot arm comprising an end effector;
• a camera configured to capture images, the camera being configured on the robot arm;
• a processing apparatus, operatively coupled to the camera, the processing device being configured to
• determine a position and an orientation of the camera relative to the end effector;
• select a first keypoint on the object;
• provide information regarding additional keypoints relative to the first keypoint in object coordinates;
• guide the end effector to the position of the first keypoint;
• determine the position and the orientation of the end effector at the first keypoint in world coordinates;
• determine the position of the first keypoint in world coordinates;
• determine the positions of the additional keypoints in world coordinates;
• alter the position and/or the orientation of the end effector and capturing an image via the camera;
• determine camera coordinates of the keypoints by converting the world coordinates of the keypoints into a camera coordinate system of the camera;
• and
• determine camera image coordinates of all keypoints using projection from the camera coordinates thereof.

A method for automated generation of ground-truth data for image annotation captured using a camera for training data of an artificial neural network, the camera being configured on a robot arm comprising an end effector, the method being performed by a processing apparatus operatively coupled to the camera and the robot arm, comprising:

• operating the robot arm and camera system to determine a position and an orientation of the camera relative to the end effector;
• selecting a first keypoint on the object;
• providing information regarding additional keypoints relative to the first keypoint in object coordinates;
• physically guiding the end effector to the position of the first keypoint;
• determining the position and the orientation of the end effector at the first keypoint in world coordinates;
• determining the position of the first keypoint in world coordinates;
• determining the positions of the additional keypoints in world coordinates;
• altering the position and/or the orientation of the end effector and capturing an image via the camera;
• determining camera coordinates of the keypoints by converting the world coordinates of the keypoints into a camera coordinate system of the camera;
• and
• determining camera image coordinates of all keypoints for the training data using projection from the camera coordinates, whereby the images are annotated without requiring physical markers or manual identification of keypoints.