Methods and systems to create a controller in an augmented reality (ar) environment using any physical object

An augmented reality (AR) system comprising:

• an image capture device configured to capture images or video of a physical object located in a real-world space;
• one or more processors;
• and
• a memory coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the AR system to perform a number of operations comprising:
• detecting at least one feature of the physical object based at least in part on the captured images or video;
• determining an orientation of the physical object in the real-world space, based at least in part on the captured images or video, without receiving control signals or communications from the physical object;
• generating, in a virtual reality (VR) environment separate and distinct from the real-world space, a virtual object representative of the physical object based at least in part on the orientation and the at least one detected feature of the physical object determined without receiving control signals or communications from the physical object;
• detecting a movement of the physical object in the real-world space, based at least in part on the captured images or video, without receiving control signals or communications from the physical object;
• and
• manipulating the virtual object in the VR environment based at least in part on the detected movement of the physical object in the real-world space.

A method performed by one or more processors of an augmented reality (AR) system, the method comprising:

• detecting at least one feature of a physical object located in a real-world space based at least in part on images or video of the physical object captured by an image capture device;
• determining an orientation of the physical object in the real-world space, based at least in part on the captured images or video, without receiving control signals or communications from the physical object;
• generating, in a virtual reality (VR) environment separate and distinct from the real-world space, a virtual object representative of the physical object based at least in part on the orientation and the at least one detected feature of the physical object determined without receiving control signals or communications from the physical object;
• detecting a movement of the physical object in the real-world space, based at least in part on the captured images or video, without receiving control signals or communications from the physical object;
• and
• manipulating one or more target objects in the real world based at least in part on the detected movement of the physical object in the real-world space.