Vision-based operation for robot

A vision-based operation method for a robot, the method comprising:

• obtaining a command for operating a tool, the command generated based on an image;
• determining a field of view of the image corresponding to a parameter;
• obtaining signals generated by sensors arranged on the robot at one or more locations including joints of the robot;
• generating a further image showing a posture of the robot based on the signals;
• and
• controlling the robot based on the command and the parameter and the further image;
• wherein the parameter is related to a size of the field of view of a camera, and wherein controlling the robot comprises:
• in response to the size decreasing, controlling the tool to move with a decreased speed;
• and
• in response to the size increasing, controlling the tool to move with an increased speed.

A vision-based operation system for a robot, comprising:

• an electronic device comprising:
• a camera;
• at least one processing unit;
• and
• a memory coupled to the at least one processing unit and storing computer program instructions therein, the instruction, when executed by the at least one processing unit, causing the electronic device to:
• obtain an image containing a tool of a robot and an object to be operated by the tool, the image being captured by the camera;
• obtain signals generated by sensors arranged on the robot including a joint of the robot;
• generate a further image showing a posture of the robot based on the signals;
• obtain a command for operating the tool, the command generated based on the image and the further image;
• determine a field of view of the image;
• control the robot based on the command and a parameter;
• and
• wherein the parameter is related to a size of the field of view of the camera,
• wherein controlling the robot comprises:
• in response to the size decreasing, controlling the tool to move with a decreased speed;
• and
• in response to the size increasing, controlling the tool to move with an increased speed;
• wherein the camera is configured to capture the image containing the tool of the robot and the object with the parameter.

A computer program product tangibly stored in a non-transient computer readable medium and including machine executable instructions which, when executed, cause a robot to obtain an image containing a tool of the robot and an object to be operated by the tool, the image being captured by a camera with a parameter;

• obtain a command for operating the tool, the command generated based on the image;
• obtain signals generated by sensors arranged on the robot including a joint of the robot;
• generate a further image showing a posture of the robot based on the signals;
• and
• control the robot based on the command and the parameter and the further image;
• wherein the parameter is related to a size of a field of view of the camera, and wherein controlling the robot comprises:
• in response to the size decreasing, controlling the tool to move with a decreased speed;
• and
• in response to the size increasing, controlling the tool to move with an increased speed.