Human-robot collaborative flexible manufacturing system and method

A system comprising:

• a processor;
• a memory having instructions stored thereon, wherein execution of the instructions by the processor causes the processor to:
• receive (i) image data from a camera mounted on a robot arm and (ii) coordinate data of a landmark on the robot arm;
• while the camera mounted on the robot arm is swept through a workspace, capture a set of images of a first workpiece and corresponding set of coordinate data of the landmark, wherein the first workpiece comprises one or more landmarks;
• determine, from the captured set of images, a surface normal orientation for the first workpiece, wherein the surface normal orientation is determined by:
• determining, by the processor, a point cloud data object of the captured set of images:
• determining, by the processor, a plane intersecting a pre-defined portion of the point cloud data object;
• and
• determining, by the processor, a normal vector or direction from the plane as the surface normal orientation;
• direct the robot arm or a mechanized actuator of the camera, to orient the camera to an orientation to capture at least one image of the first workpiece at the surface normal orientation;
• generate a composite image or a data model of the first workpiece by aggregating at least a portion of the set of images of the first workpiece, wherein the composite image or the data model includes an image of, or data object representing, the first workpiece having a workpiece region captured at a plurality of surface normal orientations;
• and
• determine using the composite image or a portion thereof and the corresponding set of coordinate data, coordinates of landmark locations of the one or more landmarks of the first workpiece and/or characteristics of the one or more landmarks;
• wherein the determined coordinates of landmark locations and/or characteristics of the one or more landmarks are used by a robotic system to perform manufacturing action on, or to, a second workpiece to make it resemble the first workpiece.

A non-transitory compute readable medium having instructions stored thereon, wherein execution of the instructions by a processor, causes the processor to:

• receive (i) image data from a camera mounted on a robot arm and (ii) coordinate data of a landmark on the robot arm;
• while the camera mounted on the robot arm is swept through a workspace, capture a set of images of a first workpiece and corresponding set of coordinate data of the landmark, wherein the first workpiece comprises one or more landmarks;
• determine, using the captured set of images, a surface normal orientation for the first workpiece, wherein the surface normal orientation is determined by:
• determining, by the processor, a point cloud data object of the captured set of images;
• determining, by the processor, a plane intersecting a pre-defined portion of the point cloud data object;
• and
• determining, by the processor, a normal vector or direction from the plane as the surface normal orientation;
• direct the robot arm or a mechanized actuator of the camera, to orient the camera to an orientation to capture at least one image of the first workpiece at the surface normal orientation;
• generate a composite image or a data model of the first workpiece by aggregating at least a portion of the set of images of the first workpiece, wherein the composite image or the data model includes an image of, or data object representing, the first workpiece having a workpiece region captured at a plurality of surface normal orientations;
• and
• determine using the composite image or a portion thereof and the corresponding set of coordinate data, coordinates of landmark locations of the one or more landmarks of the first workpiece and/or characteristics of the one or more landmarks;
• wherein the determined coordinates of landmark locations and/or characteristics of the one or more landmarks are used by a robotic system to perform manufacturing action on, or to, a second workpiece to make it resemble the first workpiece.