Adaptive tracking system for spatial input devices

A system comprising:

• a plurality of tags affixed to a plurality of objects, wherein the plurality of tags include a plurality of features such that each tag comprises at least one feature;
• a plurality of sensors, wherein a location of the plurality of sensors defines a spatial operating environment (SOE) that includes the plurality of objects;
• and
• an adaptive tracking component (ATC) running on a processor, wherein the ATC receives from each sensor of the plurality of sensors feature data corresponding to each object of the plurality of objects detected by the respective sensor, wherein the feature data is absolute three-space location data of an instantaneous state of the respective body at a point in time and space, wherein the ATC generates and maintains a coherent model of relationships between the plurality of objects and the SOE by integrating the feature data from the plurality of sensors, wherein the ATC automatically detects from the feature data a gesture of at least one object of the plurality of objects, the detecting comprising identifying the gesture using only the feature data.