Colored Point Cloud Based Refinement for Pose Estimation

A pose estimation system comprising:

• a computer system;
• a pose estimator located in the computer system, wherein the pose estimator is configured to:
• estimate initial point cloud colors for points in a colored point cloud of a surface of an object using frames in a video of the object and initial pose estimates for the object in the frames;
• adjust the initial pose estimates using the frames and the colored point cloud to form updated pose estimates;
• determine updated point cloud colors for the points in the colored point cloud using the frames in the video and the updated pose estimates;
• and
• repeat adjusting the updated pose estimates using the frames and the colored point cloud and determining the updated point cloud colors for the points in the colored point cloud using the frames in the video and the updated pose estimates until the updated pose estimates meet a threshold.

A computer program product for pose estimation, the computer program product comprising:

• a set of one or more computer-readable storage media;
• program instructions, collectively stored in the set of one or more computer-readable storage media, for causing a processor set to perform the following computer operations:
• estimate initial point cloud colors for points in a colored point cloud of a surface of an object using frames in a video of the object and initial pose estimates for the object in the frames;
• adjust the initial pose estimates using the frames and the colored point cloud to form updated pose estimates;
• determine updated point cloud colors for the points in the colored point cloud using the frames in the video and the updated pose estimates;
• and
• repeat adjusting the updated pose estimates using the frames and the colored point cloud and determining the updated point cloud colors for the points in the colored point cloud using the frames in the video and the updated pose estimates until the updated pose estimates meet a threshold.