Phone case for tracking and localization

A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a processor, cause the processor to perform operations for providing gesture user input to a user interface displayed by a head-worn device, using movement of a portable device in a device case, the operations comprising:

• capturing, by the head-worn device, an image of the device case;
• receiving, by the head-worn device from the portable device, a pose of the portable device, the pose of the portable device including a position of the portable device in an environment;
• determining, by the head-worn device, an initial relative pose between the head-worn device and the portable device based on the image and the pose of the portable device;
• determining, by the head-worn device, a visual-inertial odometry (VIO)-based pose of the head-worm device;
• transmitting, by the head-worn device to the portable device, the VIO-based pose of the head worn device:
• determining, by the portable device, a change in the initial relative pose due to relative movement of the head-worn device and the portable device using the VIO-based pose of the head worn device;
• transmitting, by the portable device to the head worn device, the change in the initial relative pose;
• and
• based on the change in the initial relative pose, updating a user interface.

A computing system comprising:

• a head-worn device;
• at least one processor and a memory storing instructions;
• and
• a device case for holding a portable device, the device case including light sources located therein,
• wherein the instructions stored by the memory, when executed by the processor, configure the system to perform operations for providing gesture user input to a user interface displayed by the head-worn device when the light sources on the device case are illuminated, the operations comprising:
• capturing, by the head-worn device, an image of the device case;
• receiving, by the head-worn device from the portable device, a pose of the portable device, the pose of the portable device including a position of the portable device in an environment;
• determining, by the head-worn device, an initial relative pose between the head-worn device and the portable device based on the image and the pose of the portable device;
• determining, by the head-worn device, a visual-inertial odometry (VIO)-based pose of the head-worn device;
• transmitting, by the head-worn device to the portable device, the VIO-based pose of the head worn device;
• determining, by the portable device, a change in the initial relative pose due to relative movement of the head-worn device and the portable device using the VIO-based pose of the head worn device;
• transmitting, by the portable device to the head worn device, the change in the initial relative pose;
• and
• based on the change in the initial relative pose, updating a user interface.