Haptic photogrammetry in robots and methods for operating the same

A method of operation of a robot, the robot comprising at least one processor, at least one actuatable end effector, and at least one haptic sensor positioned on the at least one actuatable end effector, the method comprising:

• accessing, by the at least one processor of the robot, an environment model representing an environment, the environment model comprising haptic data including at least one haptic profile of at least one object in the environment;
• touching, by the first actuatable end effector, a first object of the at least one object;
• capturing, by the at least one haptic sensor positioned on the first actuatable end effector, haptic feedback from the first object;
• determining a haptic profile of the first object based on the haptic feedback from the first object;
• determining an identification of the first object by matching the determined haptic profile to a reference haptic profile in a database of haptic profiles;
• accessing a visual profile corresponding to the first object in a database of visual profiles, based on the identification of the first object determined by matching the determined haptic profile to a reference haptic profile in a database of haptic profiles;
• and
• controlling, by the at least one processor, the robot based at least in part on the haptic data included in the environment model and based on feedback from the at least one haptic sensor.