Systems and methods for pose detection and measurement

A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

• receiving a plurality of images of the object captured from multiple viewpoints with respect to the object;
• initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images;
• performing an instance segmentation process on the plurality of images captured from multiple viewpoints;
• extracting an image patch for the object from an instance of the object detected by the image segmentation process;
• predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images including providing the extracted image patch to a keypoint detector;
• and
• computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints corresponding to three-dimensional points on the surface of a 3-D model of the object as arranged in accordance with the current pose, and as the 3-D keypoints are projected to each of the viewpoints.

One or more non-transitory computer storage media encoded with computer program instructions that when executed by a plurality of computers cause the plurality of computers to perform operations comprising:

• receiving a plurality of images of the object captured from multiple viewpoints with respect to the object;
• initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images;
• performing an instance segmentation process on the plurality of images captured from multiple viewpoints;
• extracting an image patch for the object from an instance of the object detected by the image segmentation process;
• predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images including providing the extracted image patch to a keypoint detector;
• and
• computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints corresponding to three-dimensional points on the surface of a 3-D model of the object as arranged in accordance with the current pose, and as the 3-D keypoints are projected to each of the viewpoints.