Machine learning enabled visual servoing with dedicated hardware acceleration

A system controller for visual servoing, comprising:

• a technology module comprising an accelerator processor for machine learning configured to:
• retrieve a desired pose configuration of a workpiece object being manipulated in its pose by a robotic device;
• iteratively receive from one or more sensors on or near the robotic device visual feedback information that includes a current pose configuration of the workpiece object;
• and
• iteratively execute a machine learning model trained to process the visual feedback information and determine a low dimensional pose configuration error based on a difference between the current pose configuration of the workpiece object and the desired pose configuration of the workpiece object;
• a pre-processing module configured to build a 3D CAD model of the workpiece object based on the visual feedback information on a condition that visual feedback information indicates that the object is unknown to the system controller, wherein the machine learning model is configured to use the 3D CAD model as training input to refine the machine learning model;
• and
• a servo control module configured to iteratively adapt a servo control signal to the robotic device for manipulation of the workpiece object in response to the pose configuration error.