Robot calibration for AR and digital twin

A system for calibrating an augmented reality (AR) device to a robot base reference frame, said system comprising:

• an industrial robot having a plurality of structural components;
• a robot controller in communication with the robot, said controller including a processor and memory and being configured with robot operational control software;
• and
• an augmented reality (AR) device in communication with the robot controller, said AR device having one or more camera or object sensor, position tracking sensors, a display, and a processor and memory configured to run an AR application,
• where the robot controller provides position data for all joints of the robot in a current pose to the AR device, and
• the AR device computes a 3D model of the robot in the current pose using 3D models of the structural components and the position data, captures image or sensor data of the robot in the current pose using the one or more camera or sensor, and compares the image or sensor data of the robot to the 3D model of the robot to determine a position and orientation of the AR device with respect to the robot base reference frame.

A method for calibrating an augmented reality (AR) device to a machine base reference frame, said method comprising:

• providing a three-dimensional (3D) model of each structural component of an articulated machine to the AR device;
• providing position data for all parts of the machine in a current pose from a machine controller to the AR device;
• computing a 3D model of the machine in the current pose, by the AR device, using the 3D models of the components and the position data;
• capturing image or sensor data of the machine in the current pose by one or more camera or sensor in the AR device;
• and
• comparing the image or sensor data of the machine to the 3D model of the machine to determine a position and orientation of the AR device with respect to the machine base reference frame.