# US 2024/0203115 A1 · Method for stabilizing line-of-sight falling point

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203115 A1
- Document ID: US20240203115A1
- Status: Published application
- Kind: A1
- Application number: 18/082,650
- Filed: 2022-12-16
- Published: 2024-06-20
- Earliest claimed: 2022-12-14
- Assignees: General Interface Solution Limited
- Applicants: General Interface Solution Limited
- Inventors: YU-HENG CHOU; Ting-Chun Men; Chia-Yu Cheng
- Web page: https://patentgrep.com/patent/US20240203115A1

## CPC

- G06V 10/98: Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
- G06T 7/73: using feature-based methods
- G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching}
- G06V 40/165: {using facial parts and geometric relationships}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/193: {Preprocessing; Feature extraction}
- G06V 40/197: {Matching; Classification}
- G06T 2207/30201: Face
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Disclosed is a line-of-sight detection method, including a step SA1 and a step SA2. The step SA1 performs a dynamic-threshold-to-switch action to correct the error of the data generated in the previous facial feature detection step. Furthermore, the step SA2 performs an eye-ROI-filtering action serving as a pretreatment step to filter a plurality of eye-ROI images and establish a sample set.

## Claims

### Claim 1

A line-of-sight detection method, including:
a step SA1 that performs a dynamic-threshold-to-switch action to correct errors of data generated in a previous facial feature detection step; and
a step SA2 that performs an eye-ROI-filtering action serving as a pretreatment step to filter a plurality of eye-ROI images and establish a sample set.

### Claim 2 (depends on 1)

The line-of-sight detection method of claim 1, wherein, in the step SA1, a dynamic threshold value serves as criteria for calculating a reference point of a head posture feature when the reference point switches.

### Claim 3 (depends on 1)

The line-of-sight detection method of claim 1, wherein the step SA1 is performed by calculating the normalized root mean square error (abbreviated NRMSE) based on a scene image collected, employing the NRMSE less than a first threshold value as a basis for image grouping, calculating a standard deviation of multiple facial features based on images of each group, and selecting points of the facial features having a standard deviation less than a second threshold value so as to set a dynamic threshold value.

### Claim 4 (depends on 1)

The line-of-sight detection method of claim 1, wherein the step SA2 is a pre-process for an image size and a pupil feature image, thereby making a difference between positive and negative samples and then providing an optimized discrimination effect.

### Claim 5 (depends on 1)

The line-of-sight detection method of claim 1, wherein the step SA2 is performed using a single template matching coefficient less than a third threshold value as criteria for determining a filter.

### Claim 6 (depends on 1)

The line-of-sight detection method of claim 1, wherein the step SA1 is performed by extracting key points corresponding to different angles.

### Claim 7 (depends on 3)

The line-of-sight detection method of claim 3, wherein the step SA1 is capable of reducing improper calculation of head swing caused by an error distribution of different points of the facial features; and wherein the step SA2 is capable of filtering out pupil detection errors resulted from human eye area positioning failure of different points of the facial features, thereby improving stability of line-of-sight detection.

### Claim 8

Aline-of-sight detection method, comprising:
a step S1 that executes a face detection;
a step S2 that executes a facial landmark detection;
a step SA1 that executes a dynamic-threshold-to-switch action;
a step S3 that executes a head pose estimation;
a step S4 that executes a pupil detection; and
a step S5 that executes a gaze estimation.

### Claim 9 (depends on 8)

The line-of-sight detection method of claim 8, wherein, in the step SA1, a dynamic threshold value serves as criteria for calculating a reference point of a head posture feature when the reference point switches.

### Claim 10 (depends on 8)

The line-of-sight detection method of claim 8, wherein the step SA1 is performed by calculating the normalized root mean square error (abbreviated NRMSE) based on a scene image collected, employing the NRMSE less than a first threshold value as a basis for image grouping, calculating a standard deviation of multiple facial features based on images of each group, and selecting points of the facial features having standard a deviation less than a second threshold value so as to set a dynamic threshold value.

### Claim 11 (depends on 8)

The line-of-sight detection method of claim 8, wherein the step SA2 is a pre-process for an image size and a pupil feature image, thereby making a difference between positive and negative samples and then providing an optimized discrimination effect.

### Claim 12 (depends on 8)

The line-of-sight detection method of claim 8, wherein the step SA2 is performed using a single template matching coefficient less than a third threshold value as criteria for determining a filter.

### Claim 13 (depends on 8)

The line-of-sight detection method of claim 8, wherein the step SA1 is performed by extracting key points corresponding to different angles.

### Claim 14 (depends on 10)

The line-of-sight detection method of claim 10, wherein the step SA1 is capable of reducing improper calculation of head swing caused by an error distribution of different points of the facial features; and wherein the step SA2 is capable of filtering out pupil detection errors resulted from human eye area positioning failure of different points of the facial features, thereby improving stability of line-of-sight detection.

### Claim 15

Aline-of-sight detection method, comprising:
a step S1 that performs a face detection;
a step S2 that performs a facial landmark detection;
a step SA1 that performs a dynamic-threshold-to-switch action;
a step S3 that performs a head pose estimation;
a step SA2 that performs an eye-ROI-filtering action;
a step S4 that performs a pupil detection; and
a step S5 that performs a gaze estimation.

### Claim 16 (depends on 15)

The line-of-sight detection method of claim 15, wherein, in the step SA1, a dynamic threshold value serves as criteria for calculating a reference point of a head posture feature when the reference point switches.

### Claim 17 (depends on 15)

The line-of-sight detection method of claim 15, wherein the step SA1 is performed by calculating the normalized root mean square error (abbreviated NRMSE) based on a scene image collected, employing the NRMSE less than a first threshold value as a basis for image grouping, calculating a standard deviation of multiple facial features based on images of each group, and selecting points of the facial features having a standard deviation less than a second threshold value so as to set a dynamic threshold value.

### Claim 18 (depends on 15)

The line-of-sight detection method of claim 15, wherein the step SA2 is a pre-process for an image size and a pupil feature image, thereby making a difference between positive and negative samples and then providing an optimized discrimination effect.

### Claim 19 (depends on 15)

The line-of-sight detection method of claim 15, wherein the step SA2 is performed using a single template matching coefficient less than a third threshold value as criteria for determining a filter.

### Claim 20 (depends on 17)

The line-of-sight detection method of claim 17, wherein the step SA1 is capable of reducing improper calculation of head swing caused by an error distribution of different points of the facial features; and wherein the step SA2 is capable of filtering out pupil detection errors resulted from human eye area positioning failure of different points of the facial features, thereby improving stability of line-of-sight detection.
