A system for programming a robotic device, the system comprising:
- a sensor;
- and
- a computing system communicably coupled to the sensor and configured to:
- identify, within an environment, a robotic device and a calibration fixture in a vicinity of the robotic device;
- reference the calibration fixture to a predetermined portion of the robotic device to determine a first pose of the robotic device relative to the calibration fixture;
- receive, from the sensor, a 3D image of the environment wherein the 3D image comprises the calibration fixture, the environment and an object disposed in the environment, wherein each of the object and the environment comprises a surface for processing by the robotic device;
- determine, based on the 3D image, a second pose of the calibration fixture relative to the sensor;
- determine a third pose of the robotic device relative to the sensor based on the first pose and the second pose;
- receive a plurality of trajectory points from a display interface or a device interface;
- determine a plurality of virtual trajectory points corresponding to the plurality of trajectory points based on the 3D image and the third pose,
- wherein the computing system is further configured to:
- receive a 2D image of the environment;
- provide for display of the 2D image of the environment overlaid with the virtual trajectory points;
- provide for display of the 2D image of the environment superimposed with the 3D image received from the sensor, wherein the 3D image being superimposed on the 2D image of the environment can appear visible or invisible on the display interface;
- and
- in response to changes in orientation of a display device which provides the display interface, update the display to show at least a portion of the 2D image superimposed with 3D image from a corresponding point of view,
- wherein the computing system is further configured to:
- provide the display interface for manipulating the plurality of virtual trajectory points;
- receive, from the display interface, input data indicating creation of and/or adjustment to a two-dimensional trajectory pattern that comprises the plurality of virtual trajectory points arranged in a specified pattern;
- based on the trajectory pattern created and adjusted, convert the two-dimensional trajectory pattern into a three-dimensional trajectory pattern on the display interface;
- receive, from the display interface, input data to translate and rotate the three-dimensional trajectory pattern on the display interface;
- receive, from the display interface, input data to project the three-dimensional trajectory pattern onto the surface of the object or the environment comprised in the 3D image shown on the display interface,
- wherein the computing system is further configured to:
- provide the display interface to adjust one or more properties of one or more of the plurality of virtual trajectory points, the properties being one or more selected from the group consisting of a trajectory orientation, an end-effector state, and a trajectory speed;
- provide the said display interface to create, duplicate or delete the one or more of the plurality of virtual trajectory points;
- determine one or more adjusted properties of one or more of the plurality of trajectory points that correspond to the one or more of the plurality of virtual trajectory points based on the one or more adjusted properties of one or more of the plurality of the virtual trajectory points.