Control device with holographic element

A head-mounted display device, comprising:

• an at least partially see-through display;
• a processor;
• and
• a memory holding instructions executable by the processor to:
• receive image data of a physical environment including control device;
• using the image data, generating a three dimensional model of at least a portion of the physical environment;
• receive position information of the control device within the physical environment;
• based at least in part on the position information, display a holographic element with the control device;
• based at least in part on the position information, determine that the control device is within a predetermined proximity of either a holographic object displayed via the head-mounted display device or a physical object;
• and
• based at least in part on determining that the control device is within the predetermined proximity, modify the holographic element.

A method for displaying via a head-mounted display device a holographic element with a hand-held control device, the method comprising:

• receiving image data of a physical environment including the hand-held control device;
• using the image data, generating a three dimensional model of at least a portion of the physical environment;
• receiving position information of the control device within the physical environment;
• based at least in part on the position information, displaying the holographic element with the hand-held control device;
• based at least in part on the position information, determining that the hand-held control device is within a predetermined proximity of either a holographic object displayed via the head-mounted display device or a physical object;
• and
• based at least in part on determining that the hand-held control device is within the predetermined proximity, modifying the holographic element displayed via the head-mounted display device to provide visual feedback indicating a new capability of the holographic element.