Using deep learning to determine gaze

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• generate and display a rendered environment to a user, wherein the rendered environment is associated with an application that is related to a category;
• determine a head pose of the user based on headset tracking data associated with a headset worn by the user;
• determine a hand pose of the user based on hand tracking data associated with a device held or worn by a hand of the user;
• access scene information associated with the displayed rendered environment, wherein the scene information comprises the category related to the application;
• and
• determine a predicted focal point of the user within the rendered environment by processing the head pose, the hand pose, and the scene information using a machine-learning model.

A system comprising:

• one or more processors;
• and a non-transitory memory coupled to the one or more processors comprising instructions executable by the one or more processors, the one or more processors operable when executing the instructions to:
• generate and display a rendered environment to a user, wherein the rendered environment is associated with an application that is related to a category;
• determine a head pose of the user based on headset tracking data associated with a headset worn by the user;
• determine a hand pose of the user based on hand tracking data associated with a device held or worn by a hand of the user;
• access scene information associated with the displayed rendered environment, wherein the scene information comprises the category related to the application;
• and
• determine a predicted focal point of the user within the rendered environment by processing the head pose, the hand pose, and the scene information using a machine-learning model.