Systems and methods for pose detection and measurement

A method for estimating a pose of an object comprising:

• receiving a plurality of images of the object captured from multiple viewpoints with respect to the object;
• initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images;
• predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images;
• and
• computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints associated with a 3-D model of the object as arranged in accordance with the current pose, and as projected to each of the viewpoints,
• wherein each of the plurality of differences corresponds to a different viewpoint of the viewpoints, and
• wherein each of the differences is computed for a viewpoint of the viewpoints based on a difference between:
• the plurality of 2-D keypoints associated with an image of the plurality of images corresponding to the viewpoint;
• and
• projecting the 3-D keypoints of the 3-D model arranged in the current pose to the viewpoint,
• wherein the cost function is:
• min
• R
• o
• ,
• T
• o
• ⁢
• ∑
• i
• :
• N
• ,
• j
• :
• M
• ⁢
• c
• i
• ⁢
• j
• ⁢
• 
• [
• u
• ij
• ,
• v
• i
• ⁢
• j
• ]
• -
• K
• j
• ⁡
• [
• R
• j
• ⁢
• T
• j
• ]
• ⁡
• [
• R
• o
• ⁢
• T
• o
• ]
• ⁡
• [
• x
• i
• ,
• y
• i
• ,
• z
• i
• ]
• 
• wherein Ro, To is the pose of the object, i is an index iterating through N 3-D keypoints, j is an index iterating through M viewpoints, cij is a confidence for a corresponding keypoint prediction [uij, vij] of an i-th keypoint in an image for corresponding to a j-th viewpoint, Rj, Tj, Kj are extrinsic parameters and intrinsic parameters of a j-th camera used to capture an image from a j-th viewpoint, and [xi, yi, zi] is an i-th 3-D keypoint.

A method for estimating a pose of an object comprising:

• receiving a plurality of images of the object captured from multiple viewpoints with respect to the object;
• initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images;
• predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images;
• and
• computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints associated with a 3-D model of the object as arranged in accordance with the current pose, and as projected to each of the viewpoints,
• wherein each of the plurality of differences corresponds to a different viewpoint of the viewpoints, and
• wherein each of the differences is computed for a viewpoint of the viewpoints based on a difference between:
• the plurality of 2-D keypoints associated with an image of the plurality of images corresponding to the viewpoint;
• and
• projecting the 3-D keypoints of the 3-D model arranged in the current pose to the viewpoint,
• wherein the cost function further accounts for symmetries in the object in accordance with:
• min
• R
• o
• ,
• T
• o
• ⁢
• ∑
• i
• :
• N
• ,
• j
• :
• M
• ⁢
• min
• S
• ∼
• v
• j
• ⁢
• c
• i
• ⁢
• j
• ⁢
• 
• [
• u
• ij
• ,
• v
• i
• ⁢
• j
• ]
• -
• K
• j
• ⁢
• S
• ⁡
• [
• R
• j
• ⁢
• T
• j
• ]
• ⁡
• [
• R
• o
• ⁢
• T
• o
• ]
• ⁡
• [
• x
• i
• ,
• y
• i
• ,
• z
• i
• ]
• 
• wherein Ro, To is the pose of the object, i is an index iterating through N 3-D keypoints, j is an index iterating through M viewpoints, cij is a confidence for a corresponding keypoint prediction [uij, vij] of an i-th keypoint in an image from a j-th viewpoint, Rj, Tj, Kj are extrinsic parameters and intrinsic parameters of the j-th camera used to capture the images from a j-th viewpoint, [xi, yi, zi] is an i-th 3-D keypoint, S is a transform between different symmetries of the object, and vj is the j-th view.