Method, apparatus, and system for automating operation of a machine

A robotic system for operating a machine, the machine comprising an interface having an interactive medium for use in performing a physical task using the machine, the robotic system comprising:

• a robotic arm comprising a proximal end and a distal end;
• an actuator coupled to the robotic arm at the distal end, the actuator being configured to physically interact with the interactive medium;
• a visual source configured to provide an image of the machine in a field of view;
• a processor coupled to the robotic arm and the visual source;
• and
• a memory coupled to the processor and configured to store computer executable programs comprising instructions that upon execution by the processor:
• extract, from the image, information indicating a metric for identifying the interface within the field of view, wherein the metric comprises at least one of a location, an orientation, a dimension, and a boundary of the interface in the field of view;
• identify the interface within the field of view using the information extracted from the image;
• operate the robotic arm to interact with the identified interface via the actuator and receive, from the robotic arm, a signal indicating detection of a change in the interactive medium by the actuator;
• extract information indicating a characteristic of the interactive medium based on the signal from the actuator indicating the detection of the change in the interactive medium, wherein the characteristic of the interactive medium includes position of at least one element of the interactive medium;
• and
• operate the robotic arm to actuate the at least one element of the interactive medium to perform the physical task using the machine.