US 12,008,159 B2 · Systems and methods for gaze-tracking

Abstract

Systems and methods are provided for predicting an eye gaze location of an operator of a computing device. In particular, the method generates an image grid that includes regions of interest based on a facial image. The facial image is based on a received image frame of a video stream that captures the operator using the computing device. The image grid further includes a region that indicate rotation information of the face. The method further uses a combination of trained neural networks to extract features of the regions of interest in the image grid and predict the eye gaze location on the screen of the computing device. The trained set of neural networks includes a convolutional neural network. The method optionally generate head pose pitch, roll, and yaw information to improve accuracy of predicting the location of an eye gaze.

Claims20

Dependent claims

A computer-implemented method for predicting a location of an eye gaze of an operator, the method comprising:

receiving an input image, wherein the input image depicting the operator gazing at a location on a screen of a device;
extracting a plurality of regions from the input image, wherein the plurality of regions include:
one or more eyes, and
a region indicating at least a position and a rotation of a face;
generating, based on the extracted plurality of regions, an image grid, wherein the image grid includes a plurality of image data, the plurality of image data including image data that correspond to distinct regions of the plurality of regions;
determining, based on a previously trained gaze-data prediction model and using the image grid, a location of the eye gaze, wherein the location includes a multi-dimensional location of the eye gaze associated with the screen of the device;
and
causing, based on the location, one or more actions associated with an operation of the device.

5 more elements

A system for predicting a location of an eye gaze of an operator, the system comprising:

a processor;
and
a memory storing computer-executable instructions that when executed by the processor cause the system to:
receive an input image, wherein the input image depicting the operator gazing at a location on a screen of a device;
extract a plurality of regions from the input image, wherein the plurality of regions include:
one or more eyes of the operator, and
a region indicating at least a position and a rotation of a face of the operator;
generate, based on the extracted plurality of regions, an image grid, wherein the image grid includes a plurality of image data, the plurality of image data including image data that correspond to distinct regions of the plurality of regions;
determine, based on a previously trained gaze-data prediction model and using the image grid, a location of the eye gaze, wherein the location includes a multi-dimensional location of the eye gaze associated with the screen of the device;
and
cause, based on the location, one or more actions associated with an operation of the device.

8 more elements

A computer-readable storage medium storing computer-executable instructions that when executed by a processor cause a computer system to:

receive an input image, wherein the input image depicting an operator gazing at a location on a screen of a device;
extract a plurality of regions from the input image, wherein the plurality of regions include:
one or more eyes, and
a region indicating at least a position and a rotation of a face;
generate, based on the extracted plurality of regions, an image grid, wherein the image grid includes a plurality of image data, the plurality of image data including image data that correspond to distinct regions of the plurality of regions;
determine, based on a previously trained gaze-data prediction model and using the image grid, a location of an eye gaze, wherein the location includes a multi-dimensional location of the eye gaze associated with the screen of the device;
and
cause, based on the location, one or more actions associated with an operation of the device.

5 more elements

Dependencies are inferred from the claim text.

Systems and methods for gaze-tracking

Life of this patent

Systems and methods for gaze-tracking

Life of this patent

Abstract

Claims20