# US 10,311,287 B2 · Face recognition system and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,287 B2 - Document ID: US10311287B2 - Status: Granted - Kind: B2 - Application number: 15/276,690 - Filed: 2016-09-26 - Published: 2018-02-22 - Granted: 2019-06-04 - Earliest claimed: 2016-08-17 - Assignees: Cloud Network Technology Singapore Pte. Ltd. - Applicants: Cloud Network Technology Singapore Pte. Ltd. - Inventors: Po-Cheng Chen - Web page: https://patentgrep.com/patent/US10311287B2 ## CPC - G06V 40/172: {Classification, e.g. identification} - G06V 10/82: using neural networks - G06V 40/161: {Detection; Localisation; Normalisation} - G06V 40/168: {Feature extraction; Face representation} - G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A face recognition system for recognizing face in two stages includes a processor and a camera. The camera captures a face image. The processor includes a image locating unit, an image recognition unit, a first feature extraction unit, a second feature extraction unit, and a comparison unit. The image locating unit locates a face in the face image. Facial landmarks are recognized by the image recognition unit and the first feature extraction unit extracts a deep-learning based facial feature. The second feature extraction unit extracts a hand-crafted facial feature. The comparison unit compares the deep-learning based facial feature and the hand-crafted facial feature with a predetermined facial samples library to output a result of facial comparison. A face recognition method is also provided. ## Claims ### Claim 1 A face recognition system comprising: a storage configured to store a plurality of units, the plurality of units being a collection of instructions of an application operable in the system; a camera, configured to capture a face image; and a processor configured to execute the plurality of units, the plurality of units comprising: a image locating unit, configured to locate a face area in the face image; a image recognition unit, configured to recognize and locate facial landmarks in the face area; a first feature extraction unit, configured to extract a deep-learning based facial feature from the facial landmarks; a second feature extraction unit, configured to extract a hand-crafted facial feature from the facial landmarks; and a comparison unit, configured to compare facial features with a predetermined facial samples library to output a result of facial comparison; wherein the facial features comprise the deep-learning based facial feature and the hand-crafted facial feature; wherein the predetermined facial samples library comprises a face classification unit, the face classification unit is generated by multiple deep-learning based facial features extracted from multiple facial samples; and wherein the comparison unit inputs the deep-learning based facial feature extracted by the first feature extraction unit to the face classification unit, and obtains a first comparison result and a first confidence value; and the comparison unit determines whether the first confidence value is greater than a first reference value, and the comparison unit outputs the first comparison result as the result of facial comparison in response to the first confidence value being greater than the first reference value. ### Claim 2 (depends on 1) The face recognition system of claim 1, wherein the facial landmarks comprise two eyes, a nose, a mouth, and a chin of the face image. ### Claim 3 (depends on 1) The face recognition system of claim 1, wherein the processor further comprises a face alignment unit; the face alignment unit is configured to determine whether the facial landmarks are presented at the same level, when the facial landmarks are not at the same level, the face alignment unit is further configured to align the facial landmarks to be at the same level. ### Claim 4 (depends on 1) The face recognition system of claim 1, wherein the first feature extraction unit is configured to extract the deep-learning based facial feature from the facial landmarks according to a convolutional neural network (CNN) algorithm. ### Claim 5 (depends on 1) The face recognition system of claim 1, wherein the second feature extraction unit is configured to extract the hand-crafted facial feature from the facial landmarks according to a scale invariant feature transform (SIFT) algorithm. ### Claim 6 (depends on 1) The face recognition system of claim 1, wherein the predetermined facial samples library further comprises multiple hand-crafted facial features extracted from the multiple facial samples; the multiple deep-learning based facial features extracted from the multiple facial samples generate the face classification unit according to a support vector machine (SVM) algorithm; and the multiple facial samples are captured from the same person. ### Claim 7 (depends on 1) The face recognition system of claim 1, wherein when the first confidence value is not greater than the first reference value, the comparison unit compares the hand-crafted facial feature extracted by the second feature extraction unit with the multiple hand-crafted facial features extracted from the multiple facial samples and obtains a second comparison result and a second confidence value; and when the second comparison result is equal to the first comparison result and the second confidence value is greater than a second reference value, and the comparison unit outputs the first comparison result as the result of facial comparison. ### Claim 8 (depends on 7) The face recognition system of claim 7, wherein when the second comparison result is not equal to the first comparison result or the second confidence value is not greater than the second reference value, and the comparison unit outputs a unknown as the result of facial comparison. ### Claim 9 (depends on 8) The face recognition system of claim 8, wherein value ranges of the first confidence value and the second confidence value are 0-1.0. ### Claim 10 A face recognition method, comprising: capturing a face image and locating a face area in the face image; recognizing and locating facial landmarks in the face area; extracting a deep-learning based facial feature from the facial landmarks; extracting a hand-crafted facial feature from the facial landmarks; and comparing facial features with a predetermined facial samples library to output a result of facial comparison; wherein the facial features comprise the deep-learning based facial feature and the hand-crafted facial feature; wherein the predetermined facial samples library comprises a face classification unit, the face classification unit is generated by multiple deep-learning based facial features extracted from multiple facial samples; and wherein the step of comparing facial features with a predetermined facial samples library and output a result of facial comparison comprises: inputting the deep-learning based facial feature to the face classification unit and obtaining a first comparison result and a first confidence value; determining whether the first confidence value is greater than a first reference value; and outputting the first comparison result as the result of facial comparison in response to the first confidence value being greater than the first reference value. ### Claim 11 (depends on 10) The face recognition method of claim 10, wherein the facial landmarks comprise two eyes, a nose, a mouth, and a chin of the face image. ### Claim 12 (depends on 10) The face recognition method of claim 10, wherein the step of recognizing and locating facial landmarks in the face area comprises: determining whether the facial landmarks are presented at the same level; if not, aligning the facial landmarks at the same level; and if yes, processing the extracting step. ### Claim 13 (depends on 10) The face recognition method of claim 10, wherein the step of extracting a deep-learning based facial feature from the facial landmarks comprises: extracting the deep-learning based facial feature from the facial landmarks according to a CNN algorithm. ### Claim 14 (depends on 10) The face recognition method of claim 10, wherein the step of extracting a hand-crafted facial feature from the facial landmarks comprises: extract the hand-crafted facial feature from the facial landmarks according to a SIFT algorithm. ### Claim 15 (depends on 10) The face recognition method of claim 10, wherein the predetermined facial samples library further comprises multiple hand-crafted facial features extracted from the multiple facial samples; the multiple deep-learning based facial features extracted from the multiple facial samples generate the face classification unit according to a support vector machine (SVM) algorithm; and the multiple facial samples are captured from the same person. ### Claim 16 (depends on 10) The face recognition method of claim 10, wherein the step of comparing facial features with a predetermined facial samples library and output a result of facial comparison further comprises: comparing the hand-crafted facial feature with the multiple hand-crafted facial features extracted from the multiple facial samples in response to the first confidence value being not greater than the first reference value, and obtaining a second comparison result and a second confidence value; outputting the first comparison result as the result of facial comparison in response to the second comparison result being equal to the first comparison result and the second confidence value being greater than a second reference value. ### Claim 17 (depends on 16) The face recognition method of claim 16, wherein the step of comparing facial features with a predetermined facial samples library and output a result of facial comparison further comprises: outputting a unknown as the result of facial comparison in response to the second comparison result being not equal to the first comparison result or the second confidence value being not greater than the second reference value. ### Claim 18 (depends on 17) The face recognition method of claim 17, wherein value ranges of the first confidence value and the second confidence value are 0-1.0.