# US 2021/0081653 A1 · Method and device for facial image recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0081653 A1
- Document ID: US20210081653A1
- Status: Published application
- Kind: A1
- Application number: 16/807,292
- Filed: 2020-03-03
- Published: 2021-03-18
- Earliest claimed: 2019-09-18
- Assignees: VIA Technologies, Inc.
- Applicants: Via Technologies, Inc.
- Inventors: Hao SUN
- Web page: https://patentgrep.com/patent/US20210081653A1

## CPC

- G06V 10/772: Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/2193: {based on specific statistical tests}
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/7796: {based on specific statistical tests}
- G06V 10/82: using neural networks
- G06V 40/167: {using comparisons between temporally consecutive images}
- G06V 40/172: {Classification, e.g. identification}
- G06N 3/045: Combinations of networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/08: Learning methods
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for facial image recognition is provided. A plurality of original facial images are received. A plurality of standard facial images corresponding to the original facial images are generated through a standard face generation model. A recognition model is trained by using the original facial images and the standard facial images. The recognition model is tested by using the original facial image test set and a standard facial image test set until the recognition model recognizes that the first accuracy rate of the original facial image test set is higher than a first threshold value and the second accuracy rate of the standard facial image test set is higher than a second threshold value. The original facial image test set is composed of the original facial images obtained by sampling, and the standard facial image test set is composed of the standard facial images obtained by sampling.

## Claims

### Claim 1

A method for facial image recognition, comprising:
receiving a plurality of original facial images;
generating a plurality of standard facial images corresponding to the original facial images through a standard face generation model;
training a recognition model by using the original facial images and the standard facial images; and
testing the recognition model by using an original facial image test set and a standard facial image test set until the recognition model recognizes that a first accuracy rate of the original facial image test set is higher than a first threshold value and a second accuracy rate of the standard facial image test set is higher than a second threshold value, wherein the original facial image test set is composed of the original facial images obtained by sampling, and the standard facial image test set is composed of the standard facial images obtained by sampling.

### Claim 2 (depends on 1)

The method as claimed in claim 1, further comprising:
using a hybrid test set to test the recognition model, wherein the hybrid test set consists of a random sample of the original facial images and the standard facial images.

### Claim 3 (depends on 1)

The method as claimed in claim 1, further comprising:
generating a first standard facial image corresponding to a first original facial image of a user through the standard face generating model;
generating a second standard facial image corresponding to a second original facial image of the user through the standard face generation model;
comparing the second original facial image with the first original facial image to obtain a first similarity, and comparing the second standard facial image with the first standard facial image to obtain a second similarity; and
calculating a final similarity according to the first similarity and the second similarity.

### Claim 4 (depends on 3)

The method as claimed in claim 3, wherein the final similarity FinalSimilarity is expressed as follows:
FinalSimilarity=α×SimiP2P1+β×SimiF2F1,
where SimiP2P1 is the first similarity, SimiF2F1 is the second similarity, α and β are values between 0 and 1, α+β=1 and α≥β.

### Claim 5 (depends on 3)

The method as claimed in claim 3, further comprising:
comparing the second original facial image with the first standard facial image to obtain a third similarity, and comparing the second standard facial image with the first original facial image to obtain a fourth similarity; and
calculating the final similarity according to the first similarity, the second similarity, the third similarity, and the fourth similarity.

### Claim 6 (depends on 5)

The method as claimed in claim 5, wherein the final similarity FinalSimilarity is expressed as follows:
FinalSimilarity=α×SimiP2P1+β×SimiF2F1+γ×SimiP2P1+δ×SimiF2F1,
where SimiP2P1 is the first similarity, SimiF2F1 is the second similarity, SimiP2F1 is the third similarity, and SimiF2P1 is the fourth similarity, α, β, γ and δ are values between 0 and 1, and α+β+γ+δ=1, α≥β+γ+δ, and β>γ+δ.

### Claim 7 (depends on 1)

The method as claimed in claim 1, wherein each of the standard facial images is a frontal facial image including only one face area and no other objects in the face area.

### Claim 8 (depends on 1)

The method as claimed in claim 1, wherein the original facial images are facial images having multiple expressions or other objects.

### Claim 9 (depends on 1)

The method as claimed in claim 1, wherein the standard face generation model is a generative adversarial network (GAN).

### Claim 10 (depends on 1)

The method as claimed in claim 1, wherein the standard face generation model is a variational auto-encoder (VAE).

### Claim 11

A face recognition device, comprising:
one or more processors; and
one or more computer storage media, storing computer readable instructions, wherein the processor uses the computer storage media to:
receive a plurality of original facial images;
generate a plurality of standard facial images corresponding to the original facial images through a standard face generation model;
train a recognition model using the original facial images and the standard facial images; and
test the recognition model using an original facial image test set and a standard facial image test set until the recognition model recognizes that a first accuracy rate of the original facial image test set is higher than a first threshold value and a second accuracy rate of the standard facial image test set is higher than a second threshold value, wherein the original facial image test set is composed of the original facial images obtained by sampling, and the standard facial image test set is composed of the standard facial images obtained by sampling.

### Claim 12 (depends on 11)

The face recognition device as claimed in claim 11, wherein the process further uses the computer storage media to:
use a hybrid test set to test the recognition model, wherein the hybrid test set consists of a random sample of the original facial images and the standard facial images.

### Claim 13 (depends on 11)

The face recognition device as claimed in claim 11, wherein the process further uses the computer storage media to:
generate a first standard facial image corresponding to a first original facial image of a user through the standard face generating model;
generate a second standard facial image corresponding to a second original facial image of the user through the standard face generation model;
compare the second original facial image with the first original facial image to obtain a first similarity, and compare the second standard facial image with the first standard facial image to obtain a second similarity; and
calculate a final similarity according to the first similarity and the second similarity.

### Claim 14 (depends on 13)

The face recognition device as claimed in claim 13, wherein the final similarity FinalSimilarity is expressed as follows:
FinalSimilarity=α×SimiP2P1+β×SimiF2F1,
where SimiP2P1 is the first similarity, SimiF2F1 is the second similarity, α and β are values between 0 and 1, α+β=1 and α≥β.

### Claim 15 (depends on 13)

The face recognition device as claimed in claim 13, wherein the process further uses the computer storage media to:
compare the second original facial image with the first standard facial image to obtain a third similarity, and comparing the second standard facial image with the first original facial image to obtain a fourth similarity; and
calculate the final similarity according to the first similarity, the second similarity, the third similarity, and the fourth similarity.

### Claim 16 (depends on 15)

The face recognition device as claimed in claim 15, wherein the final similarity FinalSimilarity is expressed as follows:
FinalSimilarity=α×SimiP2P1+β×SimiF2F1+γ×SimiP2P1+δ×SimiF2F1,
where SimiP2P1 is the first similarity, SimiF2F1 is the second similarity, SimiP2F1 is the third similarity, and SimiF2P1 is the fourth similarity, α, β, γ and δ are values between 0 and 1, and α+β+γ+δ=1, α≥β+γ+δ, and β>γ+δ.

### Claim 17 (depends on 11)

The face recognition device as claimed in claim 11, wherein each of the standard facial images is a frontal facial image including only one face area and no other objects in the face area.

### Claim 18 (depends on 11)

The face recognition device as claimed in claim 11, wherein the original facial images are facial images having multiple expressions or other objects.

### Claim 19 (depends on 11)

The face recognition device as claimed in claim 11, wherein the standard face generation model is a generative adversarial network (GAN).

### Claim 20 (depends on 11)

The face recognition device as claimed in claim 11, wherein the standard face generation model is a variational auto-encoder (VAE).

### Claim 21

A method for facial image recognition, comprising:
generating a first standard facial image corresponding to a first original facial image of a user through a standard face generating model;
generating a second standard facial image corresponding to a second original face image of the user through the standard face generation model;
comparing the second original facial image with the first original facial image to obtain a first similarity, and comparing the second standard facial image with the first standard facial image to obtain a second similarity; and
calculating a final similarity according to the first similarity and the second similarity.

### Claim 22 (depends on 21)

The method as claimed in claim 21, wherein the first original facial image and the first standard facial image are pre-stored images.

### Claim 23 (depends on 21)

The method as claimed in claim 21, wherein the second original facial image and the second standard facial image are instant images.

### Claim 24 (depends on 21)

The method as claimed in claim 21, wherein the final similarity FinalSimilarity is expressed as follows:
FinalSimilarity=α×SimiP2P1+β×SimiF2F1,
where SimiP2P1 is the first similarity, SimiF2F1 is the second similarity, α and β are values between 0 and 1, α+β=1 and α≥β.

### Claim 25 (depends on 24)

The method as claimed in claim 24, wherein α=0.5 and β=0.5.

### Claim 26 (depends on 21)

The method as claimed in claim 21, further comprising:
comparing the second original facial image with the first standard facial image to obtain a third similarity, and comparing the second standard facial image with the first original facial image to obtain a fourth similarity; and
calculating the final similarity according to the first similarity, the second similarity, the third similarity, and the fourth similarity.

### Claim 27 (depends on 26)

The method as claimed in claim 26, wherein the final similarity FinalSimilarity is expressed as follows:
FinalSimilarity=α×SimiP2P1+β×SimiF2F1+γ×SimiP2P1+δ×SimiF2F1,
where SimiP2P1 is the first similarity, SimiF2F1 is the second similarity, SimiP2F1 is the third similarity, and SimiF2P1 is the fourth similarity, α, β, γ and δ are values between 0 and 1, and α+β+γ+δ=1, α≥β+γ+δ, and β>γ+δ.

### Claim 28 (depends on 25)

The method as claimed in claim 25, wherein α=0.5, β=0.3, γ=0.1, and δ=0.1.

### Claim 29 (depends on 21)

The method as claimed in claim 21, wherein each of the standard facial images is a frontal facial image including only one face area and no other objects in the face area.

### Claim 30 (depends on 21)

The method as claimed in claim 21, wherein the original facial images are facial images having multiple expressions or other objects.

### Claim 31 (depends on 21)

The method as claimed in claim 21, wherein the standard face generation model is a generative adversarial network (GAN).

### Claim 32 (depends on 21)

The method as claimed in claim 21, wherein the standard face generation model is a variational auto-encoder (VAE).

### Claim 33

A face recognition device, comprising:
one or more processors; and
one or more computer storage media, storing computer readable instructions,
wherein the processor uses the computer storage media to:
generate a first standard facial image corresponding to a first original facial image of a user through a standard face generating model;
generate a second standard facial image corresponding to a second original face image of the user through the standard face generation model;
compare the second original facial image with the first original facial image to obtain a first similarity, and compare the second standard facial image with the first standard facial image to obtain a second similarity; and
calculate a final similarity according to the first similarity and the second similarity.

### Claim 34 (depends on 33)

The face recognition device as claimed in claim 33, wherein the first original facial image and the first standard facial image are pre-stored images.

### Claim 35 (depends on 33)

The face recognition device as claimed in claim 33, wherein the second original facial image and the second standard facial image are instant images.

### Claim 36 (depends on 33)

The face recognition device as claimed in claim 33, wherein the final similarity FinalSimilarity is expressed as follows:
FinalSimilarity=α×SimiP2P1+β×SimiF2F1,
where SimiP2P1 is the first similarity, SimiF2F1 is the second similarity, α and β are values between 0 and 1, α+β=1 and α≥β.

### Claim 37 (depends on 36)

The face recognition device as claimed in claim 36, wherein α=0.5 and β=0.5.

### Claim 38 (depends on 33)

The face recognition device as claimed in claim 33, wherein the process further uses the computer storage media to:
compare the second original facial image with the first standard facial image to obtain a third similarity, and comparing the second standard facial image with the first original facial image to obtain a fourth similarity; and
calculate the final similarity according to the first similarity, the second similarity, the third similarity, and the fourth similarity.

### Claim 39 (depends on 38)

The face recognition device as claimed in claim 38, wherein the final similarity FinalSimilarity is expressed as follows:
FinalSimilarity=α×SimiP2P1+β×SimiF2F1+γ×SimiP2P1+δ×SimiF2F1,
where SimiP2P1 is the first similarity, SimiF2F1 is the second similarity, SimiP2F1 is the third similarity, and SimiF2P1 is the fourth similarity, α, β, γ and δ are values between 0 and 1, and α+β+γ+δ=1, α≥β+γ+δ, and β>γ+δ.

### Claim 40 (depends on 39)

The face recognition device as claimed in claim 39, wherein α=0.5, β=0.3, γ=0.1, and δ=0.1.

### Claim 41 (depends on 33)

The face recognition device as claimed in claim 33, wherein each of the standard facial images is a frontal facial image including only one face area and no other objects in the face area.

### Claim 42 (depends on 33)

The face recognition device as claimed in claim 33, wherein the original facial images are facial images having multiple expressions or other objects.

### Claim 43 (depends on 33)

The face recognition device as claimed in claim 33, wherein the standard face generation model is a generative adversarial network (GAN).

### Claim 44 (depends on 33)

The face recognition device as claimed in claim 33, wherein the standard face generation model is a variational auto-encoder (VAE).
