Rendering information in a gaze tracking device on controllable devices in a field of view to remotely control

A system for providing control of devices to render on a user operated gaze tracking device, comprising:

• a processor;
• a machine learning module;
• and
• a computer readable storage medium having computer readable program code embodied therein that when executed by the processor performs operations, the operations comprising:
• determining a field of view from the gaze tracking device of a user based on a user position;
• providing, to the machine learning module, inputs comprising information on the user and devices in the field of view of the user;
• processing, by the machine learning module, the inputs to output at least one device of the devices in the field of view of the user and a confidence level for each of the output at least one device indicating a probability the user will remotely control that device;
• rendering an augmented reality representation of information on the output at least one device having the confidence level satisfying a confidence level requirement in a view of the gaze tracking device;
• receiving user controls to remotely control a target device comprising one of the output at least one device rendered in the gaze tracking device;
• and
• transmitting the received user controls to the target device to control the target device.

A method for providing control of devices to render on a user operated gaze tracking device, comprising:

• determining a field of view from the gaze tracking device of a user based on a user position;
• providing, to a machine learning module, inputs comprising information on the user and devices in the field of view of the user;
• processing, by the machine learning module, the inputs to output at least one device of the devices in the field of view of the user and a confidence level for each of the output at least one device indicating a probability the user will remotely control that device;
• rendering an augmented reality representation of information on the output at least one device having the confidence level satisfying a confidence level requirement in a view of the gaze tracking device;
• receiving user controls to remotely control a target device comprising one of the output at least one device rendered in the gaze tracking device;
• and
• transmitting the received user controls to the target device to control the target device.