Methods and apparatuses for applying free space inputs for surface constrained controls

A mixed-reality device, comprising:

• one or more processors communicatively coupled to:
• a display device that is at least partially transparent and is configured to display a virtual object;
• a sensor;
• and
• a touch device;
• and
• a memory device coupled to the one or more processors and storing instructions executable by the one or more processors to:
• generate virtual object data for displaying a virtual object by the display device;
• output the virtual object data to the display device;
• receive, from the sensor, interaction data corresponding to a free-space interaction by a hand of a user with the virtual object;
• generate touch input data based on the interaction data;
• and
• output the touch input data to the touch device.

A method, comprising:

• generating virtual object data for displaying a virtual object by a mixed-reality device;
• outputting the virtual object data for display by the mixed-reality device;
• receiving, by a sensor of the mixed-reality device, interaction data corresponding to a free-space interaction by a hand of a user with the virtual object;
• determining an instruction associated with the free-space interaction, wherein the instruction is further associated with a touch input of a touch device;
• and
• executing the instruction.