Hybrid pixel dynamic vision sensor tracking using ir and ambient light (or depth sensor)

A tracking system, comprising

• a processor;
• a dynamic vision sensor (DVS) operably coupled to the processor, the DVS having an array of light-sensitive elements in a known configuration, wherein the dynamic vision sensor is configured to output signals corresponding to two or more events at two or more corresponding light-sensitive elements in the array in response to changes in light output from two or more light sources in a known configuration with respect to each other and with respect to a controller body, wherein the output signals include information corresponding to times of the two or more events and locations of the two or more corresponding light-sensitive elements in the array;
• and
• one or more filters configured to selectively transmit light from the two or more light sources to one or more of the light-sensitive elements in the array and to selectively block other light from reaching one or more of the light-sensitive elements in the array, wherein the one or more filters are further configured to selectively block light from the two or more light sources from reaching one or more of the light-sensitive elements in the array and to selectively transmit the other light to one or more of the light-sensitive elements in the array,
• wherein the processor is configured to determine a position and orientation of the controller from the times of the two or more events, the locations of the two or more corresponding light-sensitive elements in the array, the known configuration of the two or more light sources with respect to each other and with respect to the controller body,
• wherein the processor is further configured to determine a position and orientation of one or more objects from signals generated by two or more light-sensitive elements resulting from other light reaching the two or more light-sensitive elements.