Determination of gaze position on multiple screens using a monocular camera

One or more non-transitory computer-readable media storing instructions executable to perform operations, the operations comprising:

• receiving a captured image from an image sensor, wherein the image sensor is part of a computing system including a screen, and wherein the captured image includes a face looking at the screen;
• determining three-dimensional (3D) locations of a plurality of facial features of the face in a camera coordinate system;
• transforming the 3D locations of the plurality of facial features to virtually rotate the face towards a virtual camera and generate normalized face image data;
• determining, using a neural network, a gaze direction and an uncertainty estimation based on the normalized face image data;
• and
• identifying a selected target area on the screen corresponding to the gaze direction.

An apparatus, comprising:

• a computer processor for executing computer program instructions;
• and
• a non-transitory computer-readable memory storing computer program instructions executable by the computer processor to perform operations comprising:
• receiving a captured image from an image sensor, wherein the image sensor is part of a computing system including a screen, and wherein the captured image includes a face looking at the screen;
• determining three-dimensional (3D) locations of a plurality of facial features of the face in a camera coordinate system;
• transforming the 3D locations of the plurality of facial features to virtually rotate the face towards a virtual camera and generate normalized face image data;
• determining, using a neural network, a gaze direction and an uncertainty estimation based on the normalized face image data;
• and
• identifying a selected target area on the screen corresponding to the gaze direction.