# US 2023/0274537 A1 · Eye gaze tracking using neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274537 A1
- Document ID: US20230274537A1
- Status: Published application
- Kind: A1
- Application number: 18/094,933
- Filed: 2023-01-09
- Published: 2023-08-31
- Earliest claimed: 2016-06-28
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Dmitry Lagun; Junfeng He; Pingmei Xu
- Web page: https://patentgrep.com/patent/US20230274537A1

## CPC

- G06V 10/82: using neural networks
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 18/211: Selection of the most significant subset of features
- G06F 18/24133: {Distances to prototypes}
- G06T 7/74: {involving reference images or patches}
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06V 10/764: using classification, e.g. of video objects
- G06V 40/19: Sensors therefor
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30201: Face
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for characterizing a gaze position of a user in a query image. One of the methods includes obtaining a query image of a user captured by a camera of a mobile device; obtaining device characteristics data specifying (ii) characteristics of the mobile device, (ii) characteristics of the camera of the mobile device, or (iii) both; and processing a neural network input comprising (i) one or more images derived from the query image and (ii) the device characteristics data using a gaze prediction neural network, wherein the gaze prediction neural network is configured to, at run time and after the gaze prediction neural network has been trained, process the neural network input to generate a neural network output that characterizes a gaze position of the user in the query image.

## Claims

### Claim 1

A method for characterizing a gaze position of a user in a query image, the method comprising:
obtaining a query image of a user captured by a camera of a mobile device;
obtaining device characteristics data specifying (ii) characteristics of the mobile device, (ii) characteristics of the camera of the mobile device, or (iii) both; and
processing a neural network input comprising (i) one or more images derived from the query image and (ii) the device characteristics data using a gaze prediction neural network, wherein the gaze prediction neural network is configured to, at run time and after the gaze prediction neural network has been trained, process the neural network input to generate a neural network output that characterizes a gaze position of the user in the query image.
