# US 2024/0004462 A1 · Gaze tracking for user interface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004462 A1
- Document ID: US20240004462A1
- Status: Published application
- Kind: A1
- Application number: 17/856,731
- Filed: 2022-07-01
- Published: 2024-01-04
- Earliest claimed: 2022-07-01
- Assignees: Sony Interactive Entertainment Inc.
- Applicants: Sony Interactive Entertainment Inc.
- Inventors: Hee Gyung Sun; Benjamin Andrew Rottler
- Web page: https://patentgrep.com/patent/US20240004462A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- A63F 13/21: characterised by their sensors, purposes or types
- A63F 13/211: using inertial sensors, e.g. accelerometers or gyroscopes
- A63F 13/428: involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
- A63F 13/533: for prompting the player, e.g. by displaying a game menu
- A63F 13/87: Communicating with other players during game play, e.g. by e-mail or chat
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G02B 27/0172: {characterised by optical features}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

Gaze tracking to track gaze of a user within a user view screen including: combining the gaze tracking with head movement tracking, wherein head movements of the head movement tracking provide rough estimate of a direction of the gaze of the user, while eye movements of the gaze tracking provide fine tuning of the direction of the gaze of the user within the user view screen; dividing the user view screen into a plurality of gaze zones when gaze zone estimation is turned on; and combining the gaze tracking with the gaze zone estimation to select a gaze zone from the plurality of gaze zones as the direction of the gaze of the user.

## Claims

### Claim 1

A method for gaze tracking to track gaze of a user within a user view screen, the method comprising:
combining the gaze tracking with head movement tracking,
wherein head movements of the head movement tracking provide rough estimate of a direction of the gaze of the user, while eye movements of the gaze tracking provide fine tuning of the direction of the gaze of the user within the user view screen;
dividing the user view screen into a plurality of gaze zones when gaze zone estimation is turned on; and
combining the gaze tracking with the gaze zone estimation to select a gaze zone from the plurality of gaze zones as the direction of the gaze of the user.

### Claim 2 (depends on 1)

The method of claim 1, further comprising
displaying an icon on the user view screen when the gaze tracking is turned on.

### Claim 3 (depends on 1)

The method of claim 1, wherein each gaze zone of the plurality of gaze zones is a preset size of a rectangular box.

### Claim 4 (depends on 1)

The method of claim 1, wherein one or more gaze zones around the selected zone is expanded or highlighted in an overlay picture or picture-in-picture mode to show a detail view of objects or persons within the gaze of the user.

### Claim 5 (depends on 1)

The method of claim 1, further comprising
displaying a notification icon on the user view screen when a notification is received.

### Claim 6 (depends on 5)

The method of claim 5, wherein the notification includes a message from a game system.

### Claim 7 (depends on 6)

The method of claim 6, wherein the notification includes a message from another user in the game system.

### Claim 8 (depends on 7)

The method of claim 7, further comprising
displaying the message on the user view screen when it is determined that the gaze of the user has moved to the notification icon.

### Claim 9

A game device comprising:
a sensor to track gaze movements of a user;
a processor operable to:
obtain information about the gaze movements from the sensor;
execute an application program to display a gaze icon based on the gaze information and a notification icon within a screen image of the application program on a display,
wherein the notification icon is displayed at a predetermined position of the screen image.

### Claim 10 (depends on 9)

The game device of claim 9, wherein the application program is a game application.

### Claim 11 (depends on 10)

The game device of claim 10, wherein the notification icon receives notification to display from application programs different from the game application.

### Claim 12 (depends on 9)

The game device of claim 9, wherein the game device is a head-mounted display (HMD).

### Claim 13

A system for gaze tracking to track gaze of a user within a user view screen, the system comprising:
a processor to combine the gaze tracking with head movement tracking, wherein head movements of the head movement tracking provide rough estimate of a direction of the gaze of the user, while eye movements of the gaze tracking provide fine tuning of the direction of the gaze of the user within the user view screen;
a gaze determiner to direct the processor to divide the user view screen into a plurality of gaze zones, and to combine the gaze tracking with the gaze zone estimation to select a gaze zone from the plurality of gaze zones as the direction of the gaze of the user, when the gaze determiner determines that gaze zone estimation is turned on.

### Claim 14 (depends on 13)

The system of claim 13, wherein the head movement tracking is performed by at least one accelerometer and at least one gyroscope disposed within a headset communicating with a game system.

### Claim 15 (depends on 14)

The system of claim 14, wherein the gaze tracking is performed by at least one camera disposed within the headset.

### Claim 16 (depends on 13)

The system of claim 13, further comprising
a user interface coupled to the processor, the user interface to display an icon on the user view screen when the gaze tracking is turned on.

### Claim 17 (depends on 16)

The system of claim 16, wherein the user interface also displays a notification icon on the user view screen when a notification is received.

### Claim 18 (depends on 17)

The system of claim 17, wherein the notification includes a message from a game system.

### Claim 19 (depends on 17)

The system of claim 17, wherein the notification includes a message from another user in a game system.

### Claim 20 (depends on 19)

The system of claim 19, wherein the game system comprises a gaming console providing interactive games.
