Augmented reality overlay

A system for object interaction in an environment of a robot, comprising:

• the robot comprising a plurality of sensors, wherein:
• a first sensor is configured for visual detection;
• and
• a second sensor is configured for spatial detection;
• a display device configured to display a virtual representation of the object within a virtual mapping of the environment based on the data received from at least one of the plurality of sensors;
• and
• a computing device configured to:
• receive data from the plurality of sensors of the robot with respect to an object within an environment of the robot;
• display a virtual representation of the object within a virtual mapping of the environment based on the data received from the plurality of sensors;
• receive input data selecting the virtual representation of the object;
• and
• send instructions the robot to act in response to the received input data.

A method for object interaction in an environment of a robot, comprising:

• receiving data from one of a plurality sensors of the robot with respect to an object within an environment of the robot;
• outputting a virtual representation of the object within the virtual mapping of the environment based on the data received from the one or more sensors;
• receiving input data selecting the virtual representation of the object;
• and
• outputting instructions to the robot to act in response to the received input data.