Method and system for facial landmark detection using facial component-specific local refinement

A system, comprising:

• at least one memory configured to store program instructions;
• at least one processor configured to execute the program instructions, which cause the at least one processor to perform steps comprising:
• performing an inference stage method, wherein the inference stage method comprises:
• receiving a first facial image;
• obtaining a first facial shape using the first facial image;
• defining, using the first facial image and the first facial shape, a plurality of facial component-specific local regions, wherein each of the facial component-specific local regions comprises a corresponding separately considered facial component of a plurality of separately considered facial components from the first facial image, and the corresponding separately considered facial component of a plurality of separately considered facial components corresponds to a corresponding first facial landmark set of the first facial landmark sets in the first facial shape, wherein the corresponding first facial landmark set of the first facial landmark sets comprises a plurality of facial landmarks;
• for each of the facial component-specific local regions, performing a cascaded regression method using each of the facial component-specific local regions and a corresponding facial landmark set of the first facial landmark sets to obtain a corresponding facial landmark set of a plurality of second facial landmark sets, wherein each stage of the cascaded regression method comprises:
• extracting a plurality of local features using each of the facial component-specific local regions and a corresponding facial landmark set of a plurality of previous stage facial landmark sets,
• wherein:
• the step of extracting comprises extracting each of the local features from a facial landmark-specific local region around a corresponding facial landmark of the corresponding facial landmark set of the previous stage facial landmark sets, wherein the facial landmark-specific local region is in each of the facial component-specific local regions;
• and
• the corresponding facial landmark set of the previous stage facial landmark sets corresponding to a beginning stage of the cascaded regression method is the corresponding facial landmark set of the first facial landmark sets;
• and
• organizing the local features based on correlations among the local features to obtain a corresponding facial landmark set of a plurality of current stage facial landmark sets, wherein the corresponding facial landmark set of the current stage facial landmark sets corresponding to a last stage of the cascaded regression method is the corresponding facial landmark set of the second facial landmark sets.