# US 2019/0318158 A1 · Multi-pose face feature point detection method based on cascade regression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318158 A1
- Document ID: US20190318158A1
- Status: Published application
- Kind: A1
- Application number: 16/469,647
- Filed: 2017-11-30
- Published: 2019-10-17
- Earliest claimed: 2016-12-14
- Assignees: South China University of Technology
- Applicants: South China University of Technology
- Inventors: Huabiao QIN; Caiman LIAO
- Web page: https://patentgrep.com/patent/US20190318158A1

## CPC

- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06F 18/23213: with fixed number of clusters, e.g. K-means clustering
- G06F 18/24323: {Tree-organised classifiers}
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7747: {Organisation of the process, e.g. bagging or boosting}
- G06V 40/165: {using facial parts and geometric relationships}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A multi-pose face feature point detection method based on cascade regression comprises: extracting a pose index features and establishing corresponding optimal weak regressors; performing corresponding initialization according to different face pose orientations; using an SIFT feature of a face image as an input feature for face orientation estimation; acquiring an orientation of an input face image according to a random forest face orientation decision tree; using a feature point mean value of a face training sample under the orientation as an initial value of the input face image feature point; and extracting the pose index feature of the face image and inputting the pose index feature into the optimal weak regressor to acquire a distribution residual to update the current feature point distribution, and complete the face feature point detection. The method can achieve a stable face feature point detection effect, and is suitable for various intelligent systems such as a face detection and recognition system, a human-computer interaction system, an expression recognition system, a driver fatigue detection system, and a gaze tracking system.

## Claims

### Claim 1

A multi-pose face feature point detection method based on cascade regression, comprising the following steps of:
(1) extracting pose index features and establishing corresponding optimal weak regressors: using a clustering algorithm to cluster face feature points to acquire feature point categories with adjacent positions; extracting pose index features under corresponding poses according to the feature point categories; and inputting the pose index features into a cascade regression algorithm, and training the pose index features to acquire the corresponding optimal weak regressors under different face poses; and
(2) performing initialization and detection on face feature points under multi-pose changes: performing corresponding initialization according to different face pose orientations; using an SIFT feature of a face image as an input feature for face orientation estimation; acquiring an orientation of an input face image according to a random forest face orientation decision tree; using a feature point mean value of a face training sample under the orientation as an initial value of the input face image feature point; and extracting the pose index feature of the face image and inputting the pose index feature into the optimal weak regressor to acquire a distribution residual to update the current feature point distribution, and complete the face feature point detection.

### Claim 2 (depends on 1)

The method according to claim 1, wherein in the step (1), the poses are divided into different training pose subsets according to the face orientation; in each pose subset, a k-means clustering algorithm is used to cluster the same category of feature points in the internal feature points of the face, and a mean value of a clustering center is continuously updated until a sum of square errors D=[*]Σi=1kΣp⊂Ci|p−mi|2 convergence, wherein C is a clustered cluster, k is a number of clusters selected according to different pose subsets, i is a face feature point, and mi is a mean value of an ith cluster; and finally, the internal feature points of the face are clustered into five categories: left eye, right eye, nose, mouth, and chin.

### Claim 3 (depends on 1)

The method according to claim 1, wherein in the step (1), the process of extracting the pose index features under different poses comprises: in training subsets with front face orientation, randomly selecting three types of candidate feature points from the five categories of left eye, right eye, nose, mouth and chin; in training subsets with left side face and left face orientations, randomly selecting three types of candidate feature points from the four categories of right eye, nose, mouth and chin; similarly, in training subsets with right side face and right face orientations, randomly selecting three types of candidate feature points from the four categories of left eye, nose, mouth and chin; after selecting corresponding feature point sets, acquiring a mean value feature of a face triangular region composed of the feature point sets.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the step of acquiring the mean value feature of the face triangular region comprises:
step a: acquiring local triangular region template sets: after selecting corresponding three types of candidate feature points in face training subsets with different poses, randomly selecting a feature point in each type as a vertex, and finally forming a closed triangular region by the three vertices, wherein the regions form N local region template sets L={Ln,n∈[1,N]};
step b: mapping reference points: mapping all templates in the set L to samples, randomly selecting a reference point Q, and determining a feature point distance by recording a distance d between the reference points and a nearest vertex Q′ in the local region to complete mapping the reference points; and
step c: solving the mean value feature of the local triangular region: after completely mapping the reference points, calculating the feature points, and acquiring a mean value
B
=
∑
j
=
1
M


I

(
Q
j
)
M
of M reference points in each closed region, wherein is a pixel value of a point Qj in the image; and differentiating N local region mean values in the set L in pairwise to acquire an N2-dimensional local region mean value feature.

### Claim 5 (depends on 1)

The method according to claim 1, wherein in the step (2), in the face orientation estimation, the SIFT feature of the input face image is extracted as a classification feature, and input into the trained random forest decision tree, that is, a probability of each category is accumulated, and output as a category with a largest sum of predicted probabilities in all the trees, and categories with different face orientation are acquired.
