Haptic photogrammetry in robots and methods for operating the same

A robot system comprising:

• at least one processor;
• a first actuatable end effector;
• at least one haptic sensor positioned on the first actuatable end effector;
• at least one non-transitory processor-readable storage medium communicatively coupled to the at least one processor, the at least one non-transitory processor-readable storage medium storing processor-executable instructions or data that, when executed by the at least one processor, cause the robot system to:
• access an environment model representing an environment, the environment model comprising haptic data including at least one haptic profile of at least one object in the environment;
• touch, by the first actuatable end effector, a first object of the at least one object;
• capture, by the at least one haptic sensor positioned on the first actuatable end effector, haptic feedback from the first object;
• determine a haptic profile of the first object based on the haptic feedback from the first object;
• determine an identification of the first object by matching the determined haptic profile to a reference haptic profile in a database of haptic profiles;
• access a visual profile corresponding to the first object in a database of visual profiles, based on the identification of the first object determined by matching the determined haptic profile to a reference haptic profile in a database of haptic profiles;
• and
• control the robot system based at least in part on the haptic data included in the environment model and based on feedback from the at least one haptic sensor.