Apparatus and method for three-dimensional object recognition

An apparatus for recognizing at least one object in a three-dimensional scene, the apparatus including an electronic processing device that:

• determines a plurality of two-dimensional images of the scene, the images at least partially including the at least one object;
• determines a plurality of two-dimensional segmentations of the at least one object, the two-dimensional segmentations corresponding to the two-dimensional images;
• generates a three-dimensional representation of the scene using the images;
• determines a pose estimation for each of the two-dimensional images;
• generates a plurality of two-dimensional representations of the three-dimensional representation using the pose estimations, each two-dimensional representation corresponding to a respective two-dimensional image;
• generates a mapping using the two-dimensional representations, the mapping being indicative of a correspondence between the images and the representation;
• and
• uses the mapping to map the plurality of segmentations to the three-dimensional representation, to thereby recognize the at least one object in the scene;
• and
• wherein each of the two-dimensional segmentations includes of a probability that each pixel in the corresponding two-dimensional image is indicative of the object and wherein the scene includes at least two objects, and the two-dimensional segmentation at least two probabilities per pixel, each probability corresponding to a respective object.

An object recognition system for recognizing at least one object in a three-dimensional scene, the system including:

• at least one image capture device for capturing a plurality of two-dimensional images;
• and, an electronic processing device that:
• determines a plurality of two-dimensional images of the scene, the images at least partially including the at least one object;
• determines a plurality of two-dimensional segmentations of the at least one object, the two-dimensional segmentations corresponding to the two-dimensional images;
• generates a three-dimensional representation of the scene using the images;
• determines a pose estimation for each of the two-dimensional images;
• generates a plurality of two-dimensional representations of the three-dimensional representation using the pose estimations, each two-dimensional representation corresponding to a respective two-dimensional image;
• generates a mapping using the two-dimensional representations, the mapping being indicative of a correspondence between the images and the representation;
• and
• uses the mapping to map the plurality of segmentations to the three-dimensional representation, to thereby recognize the at least one object in the scene;
• and
• wherein each of the two-dimensional segmentations includes of a probability that each pixel in the corresponding two-dimensional image is indicative of the object and wherein the scene includes at least two objects, and the two-dimensional segmentation at least two probabilities per pixel, each probability corresponding to a respective object.