System and method for controlling an operation of a robotic arm

A method for controlling an operation of a robotic arm holding a tool for manipulating an object, comprising:

• collecting measurements of at least one tactile sensor associated with the robotic arm;
• estimating, based on the collected measurements and constraints enforced by a Model Predictive Controller (MPC), a feedback signal indicative of a pose of the object;
• executing the MPC configured to produce, based on the estimated pose of the object, control commands for actuators of the robotic arm by optimizing a cost function minimizing a deviation of the pose of the object from a target pose of the object, wherein the optimization of the cost function is subject to the constraints, and wherein the constraints constrain one or more forces acting on the object at one or more point of contacts to be within corresponding friction regions, wherein a shape of the friction region is based on a shape of the object;
• and
• controlling the actuators of the robotic arm according to the control commands.

A non-transitory computer-readable storage medium embodied thereon a program executable by a processor for performing a method for controlling an operation of a robotic arm holding a tool for manipulating an object, the method comprising:

• collecting measurements of at least one tactile sensor associated with the robotic arm;
• estimating, based on the collected measurements and constraints enforced by a Model Predictive Controller (MPC), a feedback signal indicative of a pose of the object;
• executing the MPC configured to produce, based on the estimated pose of the object, control commands for actuators of the robotic arm by optimizing a cost function minimizing a deviation of the pose of the object from a target pose of the object, wherein the optimization of the cost function is subject to the constraints, and wherein the constraints constrain one or more forces acting on the object at one or more point of contacts to be within corresponding friction regions, wherein a shape of the friction region is based on a shape of the object;
• and
• controlling the actuators of the robotic arm according to the control commands.

A control system for controlling an operation of a robotic arm holding a tool for manipulating an object, comprising:

• at least one processor;
• and a memory having instructions stored thereon that cause the at least one processor of the control system to:
• collect measurements of at least one tactile sensor associated with the robotic arm and measurements of robot proprioception;
• compute, based on the measurements of the at least one tactile sensor, a relative orientation of frame at center of grasp of the object:
• execute, using the measurements of robot proprioception and the relative orientation of frame at center of grasp of the object, a least-squares regression to estimate the pose of the object;
• estimate, based on the collected measurements and constraints enforced by a Model Predictive Controller (MPC), a feedback signal indicative of a pose of the object;
• execute the MPC configured to produce, based on the estimated pose of the object, control commands for actuators of the robotic arm by optimizing a cost function minimizing a deviation of the pose of the object from a target pose of the object, wherein the optimization of the cost function is subject to the constraints, and wherein the constraints constrain one or more forces acting on the object at one or more point of contacts to be within corresponding friction regions;
• and
• control the actuators of the robotic arm according to the control commands.

A method for controlling an operation of a robotic arm holding a tool for manipulating an object, comprising:

• collecting measurements of at least one tactile sensor associated with the robotic arm and measurements of robot proprioception;
• computing, based on the measurements of the at least one tactile sensor, a relative orientation of frame at center of grasp of the object;
• executing, using the measurements of robot proprioception and the relative orientation of frame at center of grasp of the object, a least-squares regression to estimate the pose of the object;
• estimating, based on the collected measurements and constraints enforced by a Model Predictive Controller (MPC), a feedback signal indicative of a pose of the object;
• executing the MPC configured to produce, based on the estimated pose of the object, control commands for actuators of the robotic arm by optimizing a cost function minimizing a deviation of the pose of the object from a target pose of the object, wherein the optimization of the cost function is subject to the constraints, and wherein the constraints constrain one or more forces acting on the object at one or more point of contacts to be within corresponding friction regions;
• and
• controlling the actuators of the robotic arm according to the control commands.

A non-transitory computer-readable storage medium embodied thereon a program executable by a processor for performing a method for controlling an operation of a robotic arm holding a tool for manipulating an object, the method comprising:

• collecting measurements of at least one tactile sensor associated with the robotic arm and measurements of robot proprioception;
• computing, based on the measurements of the at least one tactile sensor, a relative orientation of frame at center of grasp of the object;
• executing, using the measurements of robot proprioception and the relative orientation of frame at center of grasp of the object, a least-squares regression to estimate the pose of the object;
• estimating, based on the collected measurements and constraints enforced by a Model Predictive Controller (MPC), a feedback signal indicative of a pose of the object;
• executing the MPC configured to produce, based on the estimated pose of the object, control commands for actuators of the robotic arm by optimizing a cost function minimizing a deviation of the pose of the object from a target pose of the object, wherein the optimization of the cost function is subject to the constraints, and wherein the constraints constrain one or more forces acting on the object at one or more point of contacts to be within corresponding friction regions;
• and
• controlling the actuators of the robotic arm according to the control commands.