# US 12,182,324 B2 · Method and apparatus for detecting delay amount of eyeball tracking apparatus, eyeball tracking system, and non-transitory computer-readable storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,324 B2
- Document ID: US12182324B2
- Status: Granted
- Kind: B2
- Application number: 17/765,708
- Filed: 2021-05-26
- Published: 2022-11-03
- Granted: 2024-12-31
- Earliest claimed: 2020-05-29
- Assignees: Beijing Boe Optoelectronics Technology Co., Ltd.; Boe Technology Group Co., Ltd.
- Applicants: Beijing BOE Optoelectronics Technology Co., Ltd.; Boe Technology Group Co., Ltd.
- Inventors: Longhui Wang; Wenyu Li; Jinghua Miao; Xi Li; Xuefeng Wang; Zihe Liu
- Web page: https://patentgrep.com/patent/US12182324B2

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06V 40/18: Eye characteristics, e.g. of the iris
- G06F 11/3041: {where the computing system component is an input/output interface (interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units G06F13/00)}
- G06F 11/3419: {by assessing time}
- G06T 7/194: involving foreground-background segmentation
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 11/00: Two-dimensional [2D] image generation
- G06V 20/20: in augmented reality scenes
- G06T 2207/10048: Infrared image
- G06T 2210/36: Level of detail
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Provided are a method and an apparatus for detecting a delay amount of an eyeball tracking apparatus, an eyeball tracking system, and a non-transitory computer-readable storage medium. The method includes: receiving a detected gaze point coordinate of a mechanical eye at the at least one detection time point; determining an actual gaze point coordinate of the mechanical eye at the at least one detection time point; and determining the delay amount of the eyeball tracking apparatus according to a coordinate difference between the actual gaze point coordinate and the detected gaze point coordinate at the at least one detection time point.

## Claims

### Claim 1

A method for detecting a delay amount of an eyeball tracking apparatus, comprising:
receiving a detected gaze point coordinate of a mechanical eye of at least one detection time point, the detected gaze point coordinate being a gaze point coordinate of the mechanical eye obtained by the eyeball tracking apparatus;
determining an actual gaze point coordinate of the mechanical eye at the at least one detection time point; and
determining the delay amount of the eyeball tracking apparatus according to a coordinate difference between the actual gaze point coordinate and the detected gaze point coordinate at the at least one detection time point,
wherein the determining the delay amount of the eyeball tracking apparatus according to the coordinate difference between the actual gaze point coordinate and the detected gaze point coordinate at the at least one detection time point comprises:
calculating the coordinate difference between the actual gaze point coordinate and the detected gaze point coordinate at the at least one detection time point; and
determining the delay amount of the eyeball tracking apparatus according to the coordinate difference and a rotation angular speed of the mechanical eye.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
presetting a mapping relationship between gaze point coordinates of the mechanical eye and time;
controlling rotation of the mechanical eye according to the mapping relationship between gaze point coordinates and time; and
obtaining the detected gaze point coordinate of the mechanical eye at the at least one detection time point during the rotation of the mechanical eye.

### Claim 3 (depends on 2)

The method of claim 2, wherein
the actual gaze point coordinate of the mechanical eye at the at least one detection time point is determined according to the mapping relationship between gaze point coordinates and time.

### Claim 4 (depends on 1)

The method of claim 1, wherein
the at least one detection time point comprises a plurality of detection time points, and the determining the delay amount of the eyeball tracking apparatus according to the coordinate difference and the rotation angular speed of the mechanical eye comprises:
determining a delay amount corresponding to each of the plurality of detection time points according to the coordinate difference and the rotation angular speed of the mechanical eye; and
determining the delay amount of the eyeball tracking apparatus according to a plurality of delay amounts corresponding to the plurality of detection time points.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
calculating an average value of the plurality of delay amounts corresponding to the plurality of detection time points, the average value being taken as the delay amount of the eyeball tracking apparatus.

### Claim 6 (depends on 5)

The method of claim 5, wherein
the delay amount of the eyeball tracking apparatus is calculated by:
d
=
1
n
⁢
∑
i
=
1
n
(
❘
"\[LeftBracketingBar]"
p
i
-
p
1
_
❘
"\[RightBracketingBar]"
*
f
⁡
(
v
)
)
wherein n is the number of the plurality of detection time points, pi is the detected gaze point coordinate of an ith detection time point, Pi is the actual gaze point coordinate of the ith detection time point, |pi−Pi| is the coordinate difference between the detected gaze point coordinate and the actual gaze point coordinate at the ith detection time point, v is the rotation angular speed of the mechanical eye, and f (v) is a function between a movement speed of the gaze point of the mechanical eye and v.

### Claim 7 (depends on 1)

The method of claim 1, wherein the obtaining the detected gaze point coordinate of the mechanical eye comprises:
emitting, by the eyeball tracking apparatus, infrared light to the mechanical eye;
acquiring, by the eyeball tracking apparatus, an infrared image of the mechanical eye; and
determining, by the eyeball tracking apparatus, the detected gaze point coordinate of the mechanical eye according to the infrared image of the mechanical eye.

### Claim 8 (depends on 1)

An apparatus for detecting a delay amount of an eyeball tracking apparatus, comprising a processor and a memory having program instructions stored thereon, wherein the processor executes the program instructions to perform the method of claim 1.

### Claim 9 (depends on 1)

A non-transitory computer-readable storage medium, having program instructions stored thereon, wherein when executed on a computer, the program instructions causes the computer to perform the method of claim 1.

### Claim 10

An apparatus for detecting a delay amount of an eyeball tracking apparatus, comprising:
a detected coordinate reception module configured to receive a detected gaze point coordinate of a mechanical eye of at least one detection time point, the detected gaze point coordinate being a gaze point coordinate of the mechanical eye obtained by the eyeball tracking apparatus;
an actual coordinate obtaining module configured to determine an actual gaze point coordinate of the mechanical eye at the at least one detection time point; and
a delay amount determination module configured to determine the delay amount of the eyeball tracking apparatus according to a coordinate difference of the actual gaze point coordinate and the detected gaze point coordinate at the at least one detection time point,
wherein the delay amount determination module comprises:
a first determination submodule configured to calculate the coordinate difference between the actual gaze point coordinate and the detected gaze point coordinate at the at least one detection time point; and
a second determination submodule configured to determine the delay amount of the eyeball tracking apparatus according to the coordinate difference and a rotation angular speed of the mechanical eye.

### Claim 11 (depends on 10)

The apparatus of claim 10, further comprising:
a control module configured to preset a mapping relationship between gaze point coordinates of the mechanical eye and time; and to control rotation of the mechanical eye according to the mapping relationship between gaze point coordinates of the mechanical eye and time, so that the detected coordinate reception module receives the detected gaze point coordinate of the mechanical eye at the at least one detection time point during the rotation of the mechanical eye.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the actual coordinate obtaining module is configured to determine the actual gaze point coordinate of the mechanical eye at the at least one detection time point according to the mapping relationship between gaze point coordinates of the mechanical eye and time.

### Claim 13 (depends on 10)

The apparatus of claim 10, wherein the at least one detection time point comprises a plurality of detection time points, and the second determination submodule is further configured to:
determine a delay amount corresponding to each of the plurality of detection time points according to the coordinate difference and the rotation angular speed of the mechanical eye; and
determine the delay amount of the eyeball tracking apparatus according to a plurality of delay amounts corresponding to the plurality of detection time points.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the second determination submodule is further configured to calculate an average value of the plurality of delay amounts corresponding to the plurality of detection time points, and to take the average value as the delay amount of the eyeball tracking apparatus.

### Claim 15 (depends on 10)

An eyeball tracking system, comprising an eyeball tracking apparatus and the apparatus for detecting the delay amount of the eye tracking apparatus of claim 10.

### Claim 16 (depends on 15)

The eyeball tracking system of claim 15, wherein the eyeball tracking apparatus is a head-mounted display device worn on a head model with the mechanical eye.

### Claim 17 (depends on 15)

The eyeball tracking system of claim 15, wherein the eyeball tracking apparatus comprises:
an emission module configured to emit infrared light to the mechanical eye;
an acquisition module configured to acquire an infrared image of the mechanical eye; and
a coordinate determination module configured to determine the detected gaze point coordinate of the mechanical eye according to the infrared image of the mechanical eye.

### Claim 18 (depends on 17)

The eyeball tracking system of claim 17, wherein the eyeball tracking apparatus further comprises:
a visual field determination module configured to determine a visual field area of the mechanical eye, with a center at the detected gaze point coordinate, according to the obtained detected gaze point coordinate of the mechanical eye;
an image processing module configured to perform image processing on a portion of an image to be displayed corresponding to the visual field area to obtain a processed image of the visual field area, and to splice a portion of the image to be displayed corresponding to other areas except the visual field area and the processed image of the visual field area, to obtain an image to be output;
an output module configured to output the image to be output; and
a display module configured to receive the image to be output by the output module and to display the image to be output.