Architecture for controlling a computer using hand gestures

A system that facilitates a user interaction with a perceptual user interface, comprising at least a processor executing the following means:

• means for tracking and detecting at least one of a plurality of objects within a scene based at least in part upon image comparison of a plurality of images relative to a course mapping of the images, the means for tracking and detecting runs multiple object hypotheses utilizing multiple trackers comprising at least a first tracker and a second tracker for tracking at least one moving object from the plurality of objects within the scene such that a second tracker tracks the at least one moving object upon failure of a first tracker to track the at least one moving object, wherein the multiple object hypotheses are supported by frame-to-frame tracking through time in at least one view and stereo matching across two views, wherein redundancy among the trackers is implemented by distance analysis, based on distance between the object and a second moving object, and wherein when the distance falls below a threshold distance the second object is removed from consideration;
• means for storing the multiple object hypotheses;
• means for iteratively seeding the tracking component with object hypotheses based upon the presence of the object characteristics and the image comparison;
• and
• means for filtering that selectively removes the tracked object from the object hypotheses and at least one object hypothesis from the set of object hypotheses, the tracked object removed based at least in part upon a region-based approach in determining depth to cursors and move windows.

A system that facilitates a user interface, comprising at least a processor executing the following components stored in a computer memory:

• a detecting component that detects at least one of a plurality of objects within a scene;
• a tracking component that tracks a detected object wherein the detected object is selected for tracking by employing multiple hypotheses on the plurality of objects within the scene wherein at least two trackers are utilized to track the detected object such that a second tracker continues tracking the detected object upon failure of a first tracker, wherein the multiple hypotheses are supported by frame-to-frame tracking through time in at least one view and stereo matching across two views, and wherein redundancy between the at least two trackers is recognized by distance analysis such that when a distance between the detected object and a second moving object falls below a threshold distance the second object is removed from consideration;
• a seeding component that iteratively seeds the tracking component with object hypotheses based upon the detected object that is tracked;
• a filtering component that selectively removes the tracked object from the object hypotheses or at least one object hypothesis from the set of object hypotheses;
• and
• an interpreting component that interprets an object characteristic of the tracked object and executes a command in response thereto.

A computer system that facilitates interaction with a user, comprising a processor executing the following components stored in computer memory:

• an object processing system for processing object information of one or more objects in a scene, the object processing system including:
• a tracking component that at least tracks the one or more objects selected by running a set of multiple hypotheses on a plurality of moving objects detected within an engagement volume, where multiple trackers comprising at least a first and second tracker are utilized to simultaneously track at least one object from the one or more tracked objects such that the second tracker continues to track the at least one object upon failure of the first tracker to track the at least one object, wherein the set of multiple hypotheses are supported by frame-to-frame tracking through time in at least one view and stereo matching across two views, and wherein when a distance between the at least an object and a second moving object falls below a threshold distance the second object is removed from consideration on an assumption that trackers tracking the at least an object and the second moving object are tracking a single object;
• a seeding component that iteratively seeds the tracking component with object hypotheses based upon the objects that are tracked;
• a filtering component that selectively removes one or more of the tracked objects and corresponding object hypothesis from the set of object hypotheses in accordance with predetermined criteria;
• and
• an interpreting component that interprets object characteristics of the at least an tracked object and causes a command to be executed in response thereto;
• an input system for receiving user input separately or in combination with the object processing system;
• and
• a presentation system for presenting information to the user in response to at least one of the command being executed and receiving user input via the input system.

A method of facilitating a human-computer interface, the method comprising:

• employing a processor to execute computer executable instructions stored in memory to perform the following acts:
• acquiring gesture characteristics of at least one of a plurality of gestures within a volume of space with an acquisition component, the gesture characteristics acquired based at least in part upon image comparison of a plurality of images relative to a course mapping of the images;
• selecting the at least one gesture by employing a set of multiple object hypotheses on the plurality of gestures within the volume of space, where multiple trackers are utilized to track each of one or more objects performing the gestures such that upon failure of a tracker to track at least one of the one or more objects, an alternate tracker continues to track the at least one object, wherein the set of multiple object hypotheses are supported by frame-to-frame tracking through time in at least one view and stereo matching across two views, and wherein when a distance between a tracked object and a second moving object falls below a threshold distance the second object is removed from consideration;
• iteratively seeding the acquisition component with at least one gesture hypothesis based upon the presence of the gesture characteristics in the volume of space and the image comparison;
• and
• programmatically controlling a graphical representation of a graphical interface in response to acquiring the at least one gesture.