Virtual interfaces for controlling IoT devices

A product control system, comprising:

• an electronic eyewear device comprising a processor, a memory, a camera, and a display;
• programming in the memory, wherein execution of the programming by the processor configures the electronic eyewear device to perform functions, including functions to:
• capture frames of video data using the camera;
• detect an Internet of Things (IoT) product at an IoT product location in a physical environment based on the frames of video data;
• estimate a current eyewear device location relative to the IoT product location;
• detect a foot shape at a foot location relative to the physical environment based on the frames of video data;
• establish a virtual plane relative to the display based on the foot location and the current eyewear device location;
• present on the display a virtual control interface along the virtual plane according to the IoT product location;
• detect the foot shape performing a foot action relative to the virtual control interface;
• and
• send a control signal to the IoT product in accordance with the foot action, wherein the control signal comprises instructions associated with an IoT action.

A non-transitory computer-readable medium storing program code which, when executed, is operative to cause an electronic processor to perform the steps of:

• capturing frames of video data using a camera, wherein the camera is coupled to an electronic eyewear device comprising the electronic processor, a memory, and a display;
• detect an Internet of Things (IoT) product at an IoT product location in a physical environment based on the frames of video data;
• estimating a current eyewear device location relative to the IoT product location;
• detecting a foot shape at a foot location relative to the physical environment based on the frames of video data;
• establishing a virtual plane relative to the display based on the foot location and the current eyewear device location;
• presenting on the display a virtual control interface along the virtual plane according to the IoT product location;
• detecting the foot shape performing a foot action relative to the virtual control interface;
• and
• sending a control signal to the IoT product in accordance with the foot action, wherein the control signal comprises instructions associated with an IoT action.