Dense 6-dof pose object detector

A method for providing training data for training an encoder-decoder convolutional artificial neural network the method comprising:

• providing, for each of a plurality of objects, a respective RGB image 2D patch corresponding to each of a plurality of poses of a repective object of the plurality of objects;
• providing, for each of the plurality of objects and for each of the plurality of poses, a corresponding ground truth ID mask patch;
• providing, for each of the plurality of objects and for each of the plurality of poses, a corresponding ground truth 2D-to-3D-correspondence map patch;
• providing a plurality of background images;
• arranging at least one of the provided RGB image 2D patches onto at least one of the plurality of background images in order to generate a sample input image,
• correspondingly arranging the corresponding ground truth 2D-to-3D-correspondence map patches onto a black background to provide a ground truth for the 2D-to-3D-correspondence map for the generated sample input image, and
• correspondingly arranging the corresponding ground truth ID mask patches onto the black background to provide a ground truth ID mask for the generated sample input image.

A System (1000) for object detection and pose estimation within an input image, the system comprising:

• an input interface for receiving the input image;
• a computing device configured to implement a trained encoder-decoder convolutional artificial neural network comprising an encoder head, an ID mask decoder head, a first correspondence color channel decoder head and a second correspondence color channel decoder head;
• wherein the ID mask decoder head is configured and trained to generate an ID mask identifying one or more objects and background in the received input image;
• wherein the first correspondence color channel decoder head is configured and trained to generate a first correspondence color channel of a 2D-to-3D-correspondence map for the one or more objects within the received input image;
• wherein the second correspondence color channel decoder head is configured and trained to generate a second correspondence color channel of the 2D-to-3D-correspondence map;
• wherein the computing device is further configured to implement a combining module and a pose determining module;
• wherein the combining module is configured to generate the 2D-to-3D-correspondence map using the generated first correspondence color channel and the generated second correspondence color channel;
• and
• wherein the pose determining module is configured to determine, for an object of the one or more objects identified by the ID mask, a pose estimation based on the generated 2D-to-3D-correspondence map and on a pre-generated bijective association of points of the object with unique value combinations in the first correspondence color channel and the second correspondence color channel.