# US 2026/0064197 A1 · Eye and face tracking with under display camera and led

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064197 A1
- Document ID: US20260064197A1
- Status: Published application
- Kind: A1
- Application number: 19/045,677
- Filed: 2025-02-05
- Published: 2026-03-05
- Earliest claimed: 2024-02-05
- Assignees: Meta Platforms Technologies, LLC
- Applicants: Meta Platforms Technologies, LLC
- Inventors: Zuoqian Wang; Min Tao; Anqi Ji; Serge Bierhuizen; Chao Han; Shizhe Shen; Adam Alan Mudrick; Jonah Shaver; Ruiqing Ma; Alexander Randon Cope
- Web page: https://patentgrep.com/patent/US20260064197A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/0172: {characterised by optical features}
- G02B 27/0179: {Display position adjusting means not related to the information to be displayed}
- G06F 3/012: {Head tracking input arrangements}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30201: Face
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A head mounted display for eye and face tracking includes an optical assembly arranged proximate to a user's eyes, a display panel arranged behind the optical assembly and visible to the user through the optical assembly along a line of sight of the user, a camera arranged within the display panel so that the user's eyes are visible to the camera through the optical assembly, and a light source arranged proximate to the camera so that light from the light source is incident upon the user's eyes through the optical assembly.

## Claims

### Claim 1

A head mounted display for eye and face tracking, comprising:
an optical assembly arranged proximate to a user's eyes;
a display panel arranged behind the optical assembly and visible to the user through the optical assembly along a line of sight of the user;
a camera arranged so that the user's eyes are visible to the camera through the optical assembly; and
a light source arranged so that light from the light source is incident upon the user's eyes through the optical assembly.

### Claim 2 (depends on 1)

The head mounted display of claim 1, wherein the camera is positioned along the line of sight of the user and oriented parallel to the line of sight of the user.

### Claim 3 (depends on 2)

The head mounted display of claim 2, wherein the light source is arranged proximate to the camera and is oriented parallel to the line of sight of the user.

### Claim 4 (depends on 1)

The head mounted display of claim 1, wherein the camera is a first camera, the light source is arranged proximate to the camera and is a first light source, and the head mounted display further comprises:
a second camera arranged within the display panel so that the user's eyes are visible to the second camera through the optical assembly; and
a second light source arranged proximate to the second camera so that light from the second light source is incident upon the user's eyes through the optical assembly,
wherein the first camera is further positioned off of the line of sight and is oriented at a first angle relative to the line of sight of the user, and
wherein the second camera is further positioned off of the line of sight and oriented at a second angle relative to the line of sight.

### Claim 5 (depends on 4)

The head mounted display of claim 4, wherein the first light source is positioned off of the line of sight and is oriented at the first angle relative to the line of sight and the second light source is positioned off of the line of sight and is oriented at the second angle relative to the line of sight.

### Claim 6 (depends on 1)

The head mounted display of claim 1, wherein the display panel is one of an LCD display panel, an LED display panel, an OLED display panel, a micro-OLED display panel, and a micro-LED display panel.

### Claim 7 (depends on 1)

The head mounted display of claim 1, wherein the display panel is an LCD display panel comprising a thin-film transistor layer attached to a back glass layer, a front glass layer, and a liquid crystal layer arranged between the back glass layer and the front glass layer, and
wherein the camera is arranged within the liquid crystal layer.

### Claim 8 (depends on 7)

The head mounted display of claim 7, wherein the light source is arranged within the liquid crystal layer.

### Claim 9 (depends on 1)

The head mounted display of claim 1, wherein the camera is positioned along the line of sight and oriented parallel to the line of sight, and the light source is positioned off of the line of sight and is oriented at an angle relative to the line of sight.

### Claim 10 (depends on 1)

The head mounted display of claim 1, wherein the display panel is one of a single display panel architecture and a dual display panel architecture.

### Claim 11

A system for eye and face tracking, comprising:
a processor;
a head mounted display in communication with the processor, the head mounted display comprising:
an optical assembly arranged proximate to a user's eyes;
a display panel arranged behind the optical assembly and visible to the user through the optical assembly along a line of sight of the user;
a camera arranged so that the user's eyes are visible to the camera through the optical assembly; and
a light source so that light from the light source is incident upon the user's eyes through the optical assembly; and
a non-transitory computer readable medium storing a set of instructions, which when executed by the processor, configure the processor to:
display to the user a real-time video on the display panel;
receive a signal from the camera;
process the signal to generate tracking data associated with the user's eyes and face; and
adjust the real-time video based on the tracking data.

### Claim 12 (depends on 11)

The system of claim 11, wherein the camera is positioned along the line of sight of the user and oriented parallel to the line of sight of the user, and the light source is arranged proximate to the camera and is oriented parallel to the line of sight of the user.

### Claim 13 (depends on 11)

The system of claim 11, wherein the camera is a first camera, the signal is a first signal, the light source is arranged proximate to the camera and is a first light source, and the head mounted display further comprises:
a second camera arranged within the display panel so that the user's eyes are visible to the second camera through the optical assembly; and
a second light source arranged proximate to the second camera so that light from the second light source is incident upon the user's eyes through the optical assembly,
wherein the first camera is further positioned off of the line of sight and is oriented at a first angle relative to the line of sight of the user,
wherein the second camera is further positioned off of the line of sight and oriented at a second angle relative to the line of sight of the user,
wherein the first light source is positioned off of the line of sight and is oriented at the first angle relative to the line of sight of the user,
wherein the second light source is positioned off of the line of sight and is oriented at the second angle relative to the line of sight of the user, and
wherein the instructions, when executed by the processor, further configure the processor to receive a second signal from the second camera, and the tracking data is generated based on the first signal and the second signal.

### Claim 14 (depends on 11)

The system of claim 11, wherein the display panel is one of an LCD display panel, an LED display panel, an OLED display panel, a micro-OLED display panel, and a micro-LED display panel.

### Claim 15 (depends on 11)

The system of claim 11, wherein the display panel is an LCD display panel comprising a thin-film transistor layer attached to a back glass layer, a front glass layer, and a liquid crystal layer arranged between the back glass layer and the front glass layer,
wherein the camera is arranged within the liquid crystal layer, and
wherein the light source is arranged within the liquid crystal layer.

### Claim 16 (depends on 11)

The system of claim 11, wherein the camera is positioned along the line of sight and oriented parallel to the line of sight, and the light source is positioned off of the line of sight and is oriented at an angle relative to the line of sight.

### Claim 17 (depends on 11)

The system of claim 11, wherein the tracking data associated with the user's eyes and face comprises at least one of eye movements of the user, gaze direction of the user, and facial expressions of the user.

### Claim 18 (depends on 11)

The system of claim 11, wherein the display panel is one of a single display panel architecture and a dual display panel architecture.

### Claim 19

A method for eye and face tracking, comprising:
displaying to a user a real-time video on a display panel arranged behind an optical assembly, the display panel visible to a user through the optical assembly along a line of sight of the user;
receiving a signal from a camera arranged within the display panel so that the user's eyes are visible to the camera through the optical assembly, the user's eyes illuminated by a light source arranged proximate to the camera so that light from the light source is incident upon the user's eyes through the optical assembly;
processing the signal to generate tracking data associated with the user's eyes and face; and
adjusting the real-time video based on the tracking data.

### Claim 20 (depends on 19)

The method of claim 19, wherein the signal is a first signal, the camera is a first camera, and the light source is a first light source, the method further comprising:
receiving a second signal from a second camera arranged within the display panel so that the user's eyes are visible to the second camera through the optical assembly, the user's eyes illuminated by a second light source arranged proximate to the second camera so that light from the second light source is incident upon the user's eyes through the optical assembly,
wherein the tracking data is generated based on the first signal and the second signal.
