Systems, methods, and control modules for controlling end effectors of robot systems

The computer-implemented method of claim 21, wherein the robot system further comprises at least one image sensor, and the method further comprises, prior to capturing the initial haptic data by the at least one haptic sensor:

• capturing, by the at least one image sensor, image data including a representation of the object;
• determining, by the at least one processor, a third state of the end effector prior to the at least one haptic sensor initially touching the object;
• predicting, by the at least one processor, the first state for the end effector to enable the at least one haptic sensor to initially touch the object;
• determining, by the at least one processor, a second transformation trajectory to transform the end effector from the third state to the first state;
• and
• controlling, by the at least one processor, the end effector to transform the end effector from the third state to the first state in accordance with the second transformation trajectory.

The computer-implemented method of claim 22, wherein:

• the robot system further comprises at least one image sensor;
• the method further comprises capturing, by the at least one image sensor, image data including a representation of the object and a representation of the end effector;
• and
• determining the first state of the end effector based at least in part on the initial haptic data comprises determining the first state based on the initial haptic data and the image data.

The computer-implemented method of claim 21, wherein:

• the first state of the end effector comprises a first position of the end effector;
• the second state of the end effector comprises a second position of the end effector different from the first position;
• determining the first transformation trajectory to transform the end effector from the first state to the second state comprises:
• determining a movement trajectory to move the end effector from the first position to the second position;
• and
• controlling the one or more degrees of freedom of the end effector to transform the end effector according to the first transformation trajectory comprises:
• controlling the end effector to move the end effector from the first position to the second position in accordance with the movement trajectory.

The computer-implemented method of claim 21, wherein:

• the first state of the end effector comprises a first orientation of the end effector;
• the second state of the end effector comprises a second orientation of the end effector different from the first orientation;
• determining the first transformation trajectory to transform the end effector from the first state to the second state comprises:
• determining a rotation trajectory to move the end effector from the first orientation to the second orientation;
• and
• controlling the one or more degrees of freedom of the end effector to transform the end effector according to the first transformation trajectory comprises:
• controlling the end effector to rotate the end effector from the first orientation to the second orientation in accordance with the rotation trajectory.

The computer-implemented method of claim 21, wherein:

• determining the first transformation trajectory to transform the end effector from the first state to the second state comprises:
• determining a first actuation trajectory to actuate the end effector from the first configuration to the second configuration;
• and
• controlling the one or more degrees of freedom of the end effector to transform the end effector according to the first transformation trajectory comprises:
• controlling the end effector to actuate the end effector from the first configuration to the second configuration in accordance with the first actuation trajectory.

A computer-implemented method of operating a robot system including an end effector having multiple degrees of freedom, at least one haptic sensor coupled to the end effector, and at least one processor, the method comprising:

• in response to the at least one haptic sensor initially touching an object, capturing initial haptic data by the at least one haptic sensor;
• determining, by the at least one processor, a first state of the end effector based at least in part on the initial haptic data, the first state comprising a first configuration of the end effector;
• determining, by the at least one processor, one or more attributes of the object;
• determining, by the at least one processor, a second state of the end effector to grasp the object based on the one or more attributes of the object and an objective, wherein the second state comprises a second configuration of the end effector that is different from the first configuration;
• determining, by the at least one processor, a first transformation trajectory to transform the end effector from the first state to the second state;
• controlling, by the at least one processor, one or more degrees of freedom of the end effector to transform the end effector according to the first transformation trajectory;
• while controlling the one or more degrees of freedom of the end effector to transform the end effector according to the first transformation trajectory, performing operations comprising:
• capturing further haptic data by the at least one haptic sensor;
• determining, by the at least one processor, at least one update to the first transformation trajectory based at least in part on the further haptic data;
• applying the at least one update to the first transformation trajectory to generate an updated first transformation trajectory;
• and
• resuming controlling the one or more degrees of freedom of the end effector based on the updated first transformation trajectory.