Neural network image processing apparatus

One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:

• identifying a face region in an image;
• determining a plurality of facial landmarks in the face region;
• determining, based at least in part on the plurality of facial landmarks, a pose of the face region;
• identifying, based at least in part on the pose, a first eye region and a second eye region within the face region;
• inputting the first eye region into a first neural network and the second eye region into a second neural network;
• receiving a first feature vector from the first neural network and a second feature vector from the second neural network;
• determining a first eyelid opening value based at least in part on the first feature vector and a second eyelid opening value based at least in part on the second feature vector;
• inputting the first eyelid opening value and the second eyelid opening value into a third neural network;
• and
• receiving, from the third neural network, a gaze yaw value or a pitch value associated with the first eyelid opening value or the second eyelid opening value.

A system comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media that, when executed by the one or more processors, cause the system to perform operations comprising:
• identifying a face region in an image;
• determining a plurality of facial landmarks in the face region;
• determining, based at least in part on the plurality of facial landmarks, a pose of the face region;
• identifying, based at least in part on the pose, a first eye region and a second eye region within the face region;
• inputting the first eye region into a first neural network and the second eye region into a second neural network;
• receiving a first feature vector from the first neural network and a second feature vector from the second neural network;
• determining a first eyelid opening value based at least in part on the first feature vector and a second eyelid opening value based at least in part on the second feature vector;
• inputting the first eyelid opening value and the second eyelid opening value into a third neural network;
• and
• receiving, from the third neural network, a gaze horizontal value or a gaze vertical value associated with the first eyelid opening value or the second eyelid opening value.