# US 2026/0064201 A1 · Corrected Gaze Direction and Origin

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064201 A1
- Document ID: US20260064201A1
- Status: Published application
- Kind: A1
- Application number: 19/380,810
- Filed: 2025-11-05
- Published: 2026-03-05
- Earliest claimed: 2022-09-23
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Julia Benndorf; Qichao Fan; Julian K. Shutzberg; Paul A. Lacey; Hua Gao
- Web page: https://patentgrep.com/patent/US20260064201A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G02B 27/0025: {for optical correction, e.g. distorsion, aberration}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/0172: {characterised by optical features}
- G06F 3/012: {Head tracking input arrangements}
- G06V 20/20: in augmented reality scenes
- G06V 40/18: Eye characteristics, e.g. of the iris
- G06V 40/193: {Preprocessing; Feature extraction}
- H04N 13/344: with head-mounted left-right displays
- G02B 2027/011: {comprising device for correcting geometrical aberrations, distortion}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods and apparatus for correcting the gaze direction and the origin (entrance pupil) in gaze tracking systems. During enrollment after an eye model is obtained, the pose of the eye when looking at a target prompt is determined. This information is used to estimate the true visual axis of the eye. The visual axis may then be used to correct the point of view (PoV) with respect to the display during use. If a clip-on lens is present, a corrected gaze axis may be calculated based on the known optical characteristics and pose of the clip-on lens. A clip-on corrected entrance pupil may then be estimated by firing two or more virtual rays through the clip-on lens to determine the intersection between the rays and the corrected gaze axis.

## Claims

### Claim 1

A device, comprising:
a display configured to be positioned in front of an eye;
an eye-facing camera configured to capture images of the eye; and
a controller comprising one or more processors configured to:
obtain a model of the eye including an optical axis and an entrance pupil;
obtain one or more images of the eye captured when the eye is looking at a target prompt on the display;
determine, from the one or more images, a difference on the display between a gaze direction of the eye when looking at the target prompt and the optical axis of the model of the eye; and
determine a visual axis for the eye based on the determined difference and the model of the eye.

### Claim 2 (depends on 1)

The device as recited in claim 1, further comprising a point of view (PoV) camera configured to capture video of an environment in front of the device, and wherein the controller is further configured to:
generate virtual content to be displayed based in part on analysis of content of the video captured by the PoV camera;
determine a current gaze direction of the eye based on the model visual axis from images of the eye captured while the virtual content is being displayed;
determine a translation from the current gaze direction to a corrected gaze direction based at least in part on the visual axis for the eye; and
determine a position on the display at which the eye is looking based on the translation.

### Claim 3 (depends on 2)

The device as recited in claim 2, further comprising a lens configured to be added to the device between the eye and the display, wherein the controller is further configured to:
detect presence of the lens in the device;
obtain optical characteristics of the lens; and
calculate a lens-corrected gaze axis by tracing a ray representing the visual axis through the lens according to the optical characteristics.

### Claim 4 (depends on 3)

The device as recited in claim 3, wherein the controller is further configured to calculate a corrected entrance pupil by tracing one or more additional virtual rays through the lens according to the optical characteristics.

### Claim 5 (depends on 3)

The device as recited in claim 3, wherein, to determine a translation from the current gaze direction to a corrected gaze direction based at least in part on the visual axis for the eye, the controller is further configured to apply a correction to the current gaze direction based on the lens-corrected gaze axis prior to applying a correction based on the PoV camera.

### Claim 6 (depends on 1)

The device as recited in claim 1, further including one or more light sources configured to emit light rays towards the eye, wherein a portion of the emitted light rays are reflected off the eye towards the eye-facing camera.

### Claim 7 (depends on 6)

The device as recited in claim 6, wherein the eye-facing camera is configured to capture images including the cornea, iris, and pupil of the eye as illuminated by the one or more light sources, wherein the captured images are processed by the controller to generate the model of the eye.

### Claim 8 (depends on 1)

The device as recited in claim 1, wherein the device is a head-mounted device (HMD) of an extended reality (XR) system.

### Claim 9

A method, comprising:
performing, by a controller comprising one or more processors:
obtaining a model of an eye including an optical axis and an entrance pupil;
receiving one or more images of the eye captured by an eye-facing camera when the eye is looking at a target prompt on a display;
determining, from the one or more images, a difference on the display between a gaze direction of the eye when looking at the target prompt and the optical axis; and
determining a visual axis for the eye based on the determined difference and the model of the eye.

### Claim 10 (depends on 9)

The method as recited in claim 9, further comprising:
generating virtual content to be displayed based in part on analysis of content of video captured by a point of view (PoV) camera;
determining a current gaze direction of the eye based on the model visual axis from images of the eye captured while the virtual content is being displayed;
determining a translation from the current gaze direction to a corrected gaze direction based at least in part on the visual axis for the eye; and
determining a position on the display at which the eye is looking based on the translation.

### Claim 11 (depends on 10)

The method as recited in claim 10, further comprising:
detecting presence of a lens between the display and the eye;
obtaining optical characteristics of the lens; and
calculating a lens-corrected gaze axis by tracing a ray representing the visual axis through the lens according to the optical characteristics.

### Claim 12 (depends on 11)

The method as recited in claim 11, further comprising calculating a corrected entrance pupil by tracing one or more additional virtual rays through the lens according to the optical characteristics.

### Claim 13 (depends on 11)

The method as recited in claim 11, wherein determining a translation from the current gaze direction to a corrected gaze direction based at least in part on the visual axis for the eye comprises applying a correction to the current gaze direction based on the lens-corrected gaze axis prior to applying a correction based on the POV camera.

### Claim 14 (depends on 9)

The method as recited in claim 9, further including one or more light sources configured to emit light rays towards the eye, wherein a portion of the emitted light rays are reflected off the eye towards the eye-facing camera.

### Claim 15 (depends on 14)

The method as recited in claim 14, wherein obtaining a model of an eye including an optical axis and an entrance pupil comprises:
receiving, from the eye-facing camera, images including the cornea, iris, and pupil of the eye as illuminated by one or more light sources; and
generating the model of the eye from the captured images.

### Claim 16 (depends on 9)

The method as recited in claim 9, wherein the controller, display, and eye-facing camera are components of a head-mounted device (HMD) of an extended reality (XR) system.

### Claim 17

A system, comprising:
a head-mounted device (HMD), comprising:
a display configured to be positioned in front of an eye;
an eye-facing camera configured to capture images of the eye;
a point of view (PoV) camera configured to capture video of an environment in front of the device; and
a controller comprising one or more processors configured to:
generate a model of the eye including an optical axis and an entrance pupil based on images of the eye captured by the eye-facing camera;
determine a visual axis for the eye based on a difference between a gaze direction of the eye when looking at a target prompt on the display and the optical axis;
generate and display virtual content based on content of the video captured by the POV camera and the eye model; and
determine a position on the display at which the eye is looking according to the visual axis.

### Claim 18 (depends on 17)

The system as recited in claim 17, further comprising a lens between the eye and the display, wherein the controller is further configured to:
obtain optical characteristics of the lens; and
calculate a lens-corrected gaze axis by tracing a ray representing the visual axis through the lens according to the optical characteristics;
wherein, to determine a position on the display at which the eye is looking according to the visual axis, the controller is further configured to apply a correction based on the lens-corrected gaze axis prior to applying a correction based on the POV camera.

### Claim 19 (depends on 18)

The system as recited in claim 18, wherein the controller is further configured to calculate a corrected entrance pupil by tracing one or more additional virtual rays through the lens according to the optical characteristics.

### Claim 20 (depends on 17)

The system as recited in claim 17, wherein the system is an extended reality (XR) system.
