Geodesic distance based primitive segmentation and fitting for 3D modeling of non-rigid objects from 2D images

A method of generating a 3D (three dimensional) geometric primitive representation of a physical cable harness as a target object for use in robotic manipulation of the cable harness, said method comprising:

• using a data processing device to implement the following steps:
• accessing a 3D point cloud representation of the target object at a given pose position;
• accessing a 2D (two dimensional) image segmentation of the target object, wherein the 2D image segmentation is generated from a 2D image of the target object at said given pose position;
• identifying a working region on said 2D image segmentation, said working region having a first pair of opposing boundaries traversing the 2D image segmentation across its principle direction, said first working region being sized to enclose all parts of the 2D image segmentation that lie between its opposing boundaries and being smaller than the total area of 2D image segmentation;
• and
• starting with said working region and repeating for a plurality of said working regions distributed along said 2D segmentation until a majority of said 2D segmentation is processed by working regions, implementing the following steps for each working region:
• (i) defining the portion of the 2D image segmentation enclosed by the working region as a 2D image sub-segment;
• (ii) defining as a cable-segment direction, a linear direction determined from a gradient of edge pixels within the 2D image sub-segment;
• (iii) within the 2D image sub-segment, identifying, as first and second key points, pixels having the largest distance perpendicular to the cable-segment direction and still bound by the 2D image sub-segment;
• (iv) identifying first and second key 3D points that respectively correspond to the first and second key points on the 2D image sub-segment;
• (v) identifying a working 3D sub-cloud comprised of 3D points from the subset of the 3D point cloud that corresponds to the 2D image sub-segment;
• (vi) fitting a tangent plane to the working 3D sub-cloud;
• (vii) projecting 3D points of the working 3D sub-cloud that are between the first and second key 3D points onto the tangent plane in the normal direction of the tangent plane;
• (viii) defining an axis line parallel to the principle direction of the projected 3D points on the tangent plane;
• (xi) projecting 3D points of the working 3D sub-cloud that are between the first and second key 3D points to the axis line;
• (x) using the first and second key 3D points to define a length for a most-current 3D geometric primitive shape oriented along the axis line and encompassing at least a portion of the working 3D sub-cloud;
• and
• (xi) using the projected 3D points within the working 3D sub-cloud to the axis line to define at least one of a width, height and radius of the most-current 3D geometric primitive shape.