Systems and methods for remapping three-dimensional gestures onto a finite-size two-dimensional surface

A real-time gesture based interactive system comprising:

• a processor;
• an acquisition system configured to capture a sequence of frames of data for constructing a depth map of a field of view of the acquisition system;
• memory storing an interaction medium tracking application, the interaction medium tracking application configuring the processor to:
• obtain the data from the acquisition system;
• compare successive frames of the frames of data for portions that change from one frame to the next;
• determine whether any of the portions that changed are part of an interaction medium detected in the sequence of frames of data;
• define an inner 3D interaction zone relative to an initial position of the part of the interaction medium detected in the sequence of frames of data, where the inner 3D interaction zone corresponds to a bounded region that is less than the frame of data and that contains the part of the interaction medium detected in the sequence of frames of data;
• track a movement of the interaction medium in the data to generate a plurality of 3D positions of the interaction medium;
• compute a 3D velocity of the interaction medium within the inner 3D interaction zone based on the 3D positions;
• compute a plurality of 2D positions based on the 3D positions and the 3D velocity, differences in the 2D positions being non-linear with respect to differences in the 3D positions;
• and
• supply the 2D positions to control an application.

A method for operating a real-time gesture based interactive system, the method comprising:

• obtaining a sequence of frames of data from an acquisition system configured to capture data for constructing a depth map of a field of view of the acquisition system;
• comparing, by a processor, successive frames the frames of data for portions that change from one frame to the next;
• determining, by the processor, whether any of the portions that changed are part of an interaction medium detected in the sequence of frames of data;
• defining, by the processor, an inner 3D interaction zone relative to an initial position of the part of the interaction medium detected in the sequence of frames of data, where the inner 3D interaction zone corresponds to a bounded region that is less than the frame of data and that contains the part of the interaction medium detected in the sequence of frames of data;
• tracking, by the processor, a movement of the interaction medium in the data to generate a plurality of 3D positions of the interaction medium;
• detecting, by the processor, movement of the interaction medium from inside to outside the inner 3D interaction zone at a boundary 3D position;
• shifting, by the processor, the inner 3D interaction zone relative to the boundary 3D position;
• computing, by the processor, a plurality of 2D positions based on the 3D positions;
• and
• supplying the 2D positions to control an application.

A method for tracking a gesture comprising:

• obtaining a sequence of frames of data from an acquisition system configured to capture data for constructing a depth map of a field of view of the acquisition system;
• comparing, by a processor, successive frames of the frames of data for portions that change from one frame to the next;
• determining, by the processor, whether any of the portions that changed are part of an interaction medium detected in the sequence of frames of data;
• defining, by the processor, an inner 3D interaction zone relative to an initial position of the part of the interaction medium detected in the sequence of frames of data, where the inner 3D interaction zone corresponds to a bounded region that is less than the frame of data and that contains the part of the interaction medium detected in the sequence of frames of data;
• tracking, by the processor, a movement of the interaction medium in the data to generate a plurality of 3D positions of the interaction medium;
• computing, by the processor, a 3D velocity of the interaction medium within the inner 3D interaction zone based on the 3D positions;
• computing, by the processor, a plurality of 2D positions based on the 3D positions and the 3D velocity, differences in the 2D positions being non-linear with respect to differences in the 3D positions;
• and
• supplying, by the processor, the 2D positions to control an application.