Camera and end-effector planning for visual servoing

A system comprising:

• processing circuitry;
• and
• memory including instructions, which when executed by the processing circuitry, cause the processing circuitry to:
• generate a set of paths of an end effector of a robotic arm by attempting to satisfy a visual-servoing task to provide a set of velocities in an end-effector task-space and satisfying a visibility-aware planning task to provide a velocity in a camera task-space, respective paths of the set of paths each including at least one respective trajectory to traverse the respective paths from a first position to a respective end position;
• determine a ranking of the set of paths using an objective function that results in a decrease in a distance between the end effector and a target while maintaining a collision mitigation path for the end effector and minimizing occlusion of a camera affixed to the end effector;
• convert a path of the set of paths into a trajectory based on the ranking, the trajectory being one of the at least one respective trajectory;
• and
• output the trajectory for to cause control of the robotic arm to move the end effector.

An apparatus comprising:

• means for generating a set of paths of an end effector of a robotic arm by attempting to satisfy a visual-servoing task to provide a set of velocities in an end-effector task-space and satisfying a visibility-aware planning task to provide a velocity in a camera task-space, respective paths of the set of paths each including at least one respective trajectory to traverse the respective paths from a first position to a respective end position;
• means for ranking the set of paths using an objective function that results in an a decrease in a distance between the end effector and a target while maintaining a collision mitigation path for the end effector and minimizing occlusion of a camera affixed to a joint of the robotic arm;
• means for converting a path of the set of paths into a trajectory based on the ranking, the trajectory being one of the at least one respective trajectory;
• and
• means for outputting the trajectory to cause control of the robotic arm to move the end effector.