# US 2023/0274485 A1 · Eyes tracking system, method and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274485 A1
- Document ID: US20230274485A1
- Status: Published application
- Kind: A1
- Application number: 18/018,085
- Filed: 2020-07-29
- Published: 2023-08-31
- Earliest claimed: 2020-07-29
- Assignees: Nippon Telegraph and Telephone Corporation
- Applicants: Nippon Telegraph and Telephone Corporation
- Inventors: Motohiro Makiguchi; Masanori YOKOYAMA; Kimitaka TSUTSUMI; Takashi SATO
- Web page: https://patentgrep.com/patent/US20230274485A1

## CPC

- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 19/003: {Navigation within 3D models or images}
- G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
- G06T 2207/30201: Face
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

There is provided a line-of-sight tracking system including a pseudo eyeball model generation device 10 and a display device 20, in which the pseudo eyeball model generation device 10 generates a pseudo eyeball model representing a pseudo eyeball with pseudo pupil information and virtual eyeball information indicating a virtual eyeball when a type of line-of-sight tracking is one of with tracking, excessive tracking, weak tracking, and no tracking, the pseudo pupil information being obtained by moving a position of a pseudo pupil back and forth when viewed from an observer with reference to a center of the virtual eyeball in association with the type of the line-of sight tracking and an intensity of the line-of-sight tracking, and the display device 20 displays the pseudo eyeball which is different according to a viewpoint position of the observer based on the pseudo eyeball model.

## Claims

### Claim 1

A line-of-sight tracking system comprising:
a display device, and
one or more processors configured to:
generate a pseudo eyeball model representing a pseudo eyeball with pseudo pupil information and virtual eyeball information indicating a virtual eyeball when a type of line-of-sight tracking is one of with tracking, excessive tracking, weak tracking, and no tracking, the pseudo pupil information being obtained by moving a position of a pseudo pupil back and forth when viewed from an observer with reference to a center of the virtual eyeball in association with the type of the line-of sight tracking and an intensity of the line-of-sight tracking, and
the display device is configured to display a pseudo eyeball image representing the pseudo eyeball which is different according to a viewpoint position of the observer based on the pseudo eyeball model.

### Claim 2 (depends on 1)

The line-of-sight tracking system according to claim 1, wherein the one or more processors are configured to determine a diameter of the pseudo pupil based on a ratio of a distance between a viewpoint of the observer located in front of the pseudo eyeball and a center of the pseudo pupil to a distance between the viewpoint and a surface of the virtual eyeball on a side of the observer.

### Claim 3 (depends on 2)

The line-of-sight tracking system according to claim 2, wherein the one or more processors are configured to determine the diameter of the pseudo pupil in consideration of an angle formed by a horizontal line passing through the center of the virtual eyeball and the viewpoint.

### Claim 4 (depends on 3)

The line-of-sight tracking system according to claim 3, wherein the display device is configured to receive emotion information indicating an emotion as an input, and display the pseudo eyeball and a facial expression corresponding to the emotion information.

### Claim 5

A line-of-sight tracking method performed by a line-of-sight tracking system including one or more processors and a display device, the method comprising:
generating, by the one or more processors, a pseudo eyeball model representing a pseudo eyeball with pseudo pupil information and virtual eyeball information indicating a virtual eyeball when a type of line-of-sight tracking is one of with tracking, excessive tracking, weak tracking, and no tracking, the pseudo pupil information being obtained by moving a position of a pseudo pupil back and forth when viewed from an observer with reference to a center of the virtual eyeball in association with the type of the line-of sight tracking and an intensity of the line-of-sight tracking; and
displaying, by the display device, a pseudo eyeball image representing the pseudo eyeball which is different according to a viewpoint position of the observer based on the pseudo eyeball model.

### Claim 6 (depends on 1)

A program for causing a computer to function as the line-of-sight tracking system according to claim 1.

### Claim 7

A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
generating, by one or more processors, a pseudo eyeball model representing a pseudo eyeball with pseudo pupil information and virtual eyeball information indicating a virtual eyeball when a type of line-of-sight tracking is one of with tracking, excessive tracking, weak tracking, and no tracking, the pseudo pupil information being obtained by moving a position of a pseudo pupil back and forth when viewed from an observer with reference to a center of the virtual eyeball in association with the type of the line-of sight tracking and an intensity of the line-of-sight tracking; and
displaying, by a display device, a pseudo eyeball image representing the pseudo eyeball which is different according to a viewpoint position of the observer based on the pseudo eyeball model.

### Claim 8 (depends on 5)

The line-of sight tracking method according to claim 5, comprising:
determining, by the one or more processors, a diameter of the pseudo pupil based on a ratio of a distance between a viewpoint of the observer located in front of the pseudo eyeball and a center of the pseudo pupil to a distance between the viewpoint and a surface of the virtual eyeball on a side of the observer.

### Claim 9 (depends on 8)

The line-of sight tracking method according to claim 8, comprising:
determining, by the one or more processors, the diameter of the pseudo pupil in consideration of an angle formed by a horizontal line passing through the center of the virtual eyeball and the viewpoint.

### Claim 10 (depends on 9)

The line-of sight tracking method according to claim 9, comprising:
receiving, by the one or more processors, emotion information indicating an emotion as an input; and
displaying, by the display device, the pseudo eyeball and a facial expression corresponding to the emotion information.

### Claim 11 (depends on 7)

The non-transitory computer-readable medium according to claim 7, comprising:
determining, by the one or more processors, a diameter of the pseudo pupil based on a ratio of a distance between a viewpoint of the observer located in front of the pseudo eyeball and a center of the pseudo pupil to a distance between the viewpoint and a surface of the virtual eyeball on a side of the observer.

### Claim 12 (depends on 11)

The non-transitory computer-readable medium according to claim 11, comprising:
determining, by the one or more processors, the diameter of the pseudo pupil in consideration of an angle formed by a horizontal line passing through the center of the virtual eyeball and the viewpoint.

### Claim 13 (depends on 12)

The non-transitory computer-readable medium according to claim 12, comprising:
receiving, by the one or more processors, emotion information indicating an emotion as an input; and
displaying, by the display device, the pseudo eyeball and a facial expression corresponding to the emotion information.
