System and methods for providing a three-dimensional touch screen

A non-transitory computer-readable medium including contents that are configured, when executed, to cause a computing system to perform a method comprising:

• managing user interaction in three dimensions with respect to a displayed user interface screen, by:
• displaying the user interface screen via a display device, wherein the user interface screen is displayed on a surface;
• performing a calibration process by:
• detecting position and orientation of the displayed user interface screen based on a pattern displayed onto the surface;
• and
• building a three-dimensional model of the surface on which the user interface screen is displayed;
• receiving position information from a visual sensor, the position information reflecting the position of an object in a three-dimensional space located adjacent to the displayed user interface screen, wherein the three-dimensional space is located between the displayed user interface screen and the visual sensor;
• generating a user interface event based on the received position information;
• and
• providing the generated user interface event to an application program.

A system comprising:

• a first computing device comprising a processor and a memory;
• and
• wherein the memory stores a code module that is configured, when executed by the processor, to manage user interaction in three dimensions with respect to a displayed user interface screen, by:
• displaying the user interface screen via a display device, wherein the user interface screen is displayed on a surface;
• performing a calibration process by:
• detecting position and orientation of the displayed user interface screen based on a pattern displayed onto the surface;
• and
• building a three-dimensional model of the surface on which the user interface screen is displayed;
• receiving position information from a visual sensor, the position information representing the position of an object in a three-dimensional space located adjacent to the displayed user interface screen, wherein the three-dimensional space is located between the displayed user interface screen and the visual sensor;
• generating a user interface event based on the received position information;
• and
• providing the generated user interface event to an application program.