Controller tracking for multiple degrees of freedom

A system for performing controller tracking, comprising:

• a depth sensor to capture, at an electronic device, images of a reflection of a modulated light pattern and to generate depth data from a local environment proximate the electronic device based on the images, the depth data comprising data corresponding to distances of spatial features of objects in the local environment from the electronic device;
• a hand-held controller including a touchpad configured to detect a contact position of a user's thumb;
• and
• a processor to:
• determine a pose of the electronic device based on the depth data;
• generate a set of positional data representing three degrees-of-freedom (3DoF) translation of the hand-held controller within the local environment based on the pose of the electronic device relative to at least one spatial feature associated with the hand-held controller;
• receive a set of rotational data representing a 3DoF orientation of the hand-held controller within the local environment;
• track a 6DoF position of the hand-held controller within the local environment based on the set of positional data and the set of rotational data;
• determine an offset of the contact position from a center axis of the touchpad;
• and
• determine the 6DoF position of the hand-held controller based on a corrected pose of the electronic device that accounts for the offset of the contact position.

A non-transitory computer readable medium having stored therein instructions, which when executed by a processor, perform operations comprising:

• generating depth data based on images of a reflection of a modulated light pattern captured at an electronic device of a local environment proximate the electronic device, the depth data comprising data corresponding to distances of spatial features of objects in the local environment from the electronic device;
• determining a pose of the electronic device based on the depth data;
• generating, based on the pose of the electronic device relative to at least one spatial feature associated with a controller, a set of positional data representing three degrees-of-freedom (3DoF) translation of the controller within the local environment;
• receiving a set of rotational data representing 3DoF orientation of the controller within the local environment;
• tracking a six degrees-of-freedom (6DoF) position of the controller within the local environment based on the set of positional data and the set of rotational data;
• receiving a set of touchpad information from the controller, wherein the set of touchpad information includes a contact position of a thumb of a user's hand on a touchpad of the controller;
• and
• determining the 6DoF position of the controller based on a corrected pose of the electronic device relative to the thumb contact position, wherein the corrected pose accounts for an offset of the contact position from a center axis of the touchpad.