Tracking wearable device and handheld object poses

A head-mounted device, comprising:

• an image sensor;
• a logic device configured to execute instructions;
• and
• a storage device comprising instructions executable by the logic device to
• receive from the image sensor an environmental tracking exposure and a handheld object tracking exposure, wherein the environmental tracking exposure is taken with different optical settings of the image sensor than the handheld object tracking exposure,
• detect in the handheld object tracking exposure a light pattern from a set of visible light sources of the plurality of visible light sources on the handheld object,
• determine a pose of the handheld object with respect to the head-mounted device based on the light pattern determined from the handheld object tracking exposure received from the image sensor of the head-mounted device,
• determine a pose of the head-mounted device with respect to a surrounding environment based upon the environmental tracking exposure received from the image sensor of the head-mounted device,
• receive from the handheld object motion sensor data comprising data related to position and orientation of the handheld object,
• fuse data representing the pose of the head-mounted device with respect to the surrounding environment and data representing the pose of the handheld object with respect to the head-mounted device with the handheld object motion sensor data to derive a pose of the handheld object relative to the surrounding environment, and
• output the pose of the handheld object relative to the surrounding environment for controlling a user interface displayed on the head-mounted device.

A head-mounted device, comprising:

• an image sensor;
• a logic device configured to execute instructions;
• and
• a storage device comprising instructions executable by the logic device to
• receive from the image sensor environmental tracking exposures and handheld object tracking exposures, wherein the environmental tracking exposures are taken with different optical settings of the image sensor than the handheld object tracking exposures,
• detect a first set of visible light sources on a first handheld object based upon the handheld object tracking exposures,
• detect a second set of visible light sources on a second handheld object based upon the handheld object tracking exposures,
• determine a pose of the first handheld object with respect to a surrounding environment based on the first set of visible light sources detected based upon the handheld object tracking exposures,
• determine a pose of the second handheld object with respect to the surrounding environment based on the second set of visible light sources detected based upon the handheld object tracking exposures,
• determine a pose of the head-mounted device with respect to a surrounding environment based upon the environmental tracking exposures received from the image sensor of the head-mounted device, and
• output the pose of the first handheld object and the pose of the second handheld object for controlling a user interface displayed on the head-mounted device.