# US 2019/0244010 A1 · Method and apparatus for registering face, and method and apparatus for recognizing face > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0244010 A1 - Document ID: US20190244010A1 - Status: Published application - Kind: A1 - Application number: 16/390,164 - Filed: 2019-04-22 - Published: 2019-08-08 - Earliest claimed: 2014-12-02 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Jungbae Kim; Seon Min Rhee; Youngkyoo Hwang; Jaejoon Han - Web page: https://patentgrep.com/patent/US20190244010A1 ## CPC - G06V 40/165: {using facial parts and geometric relationships} - G06V 20/647: {by matching two-dimensional images to three-dimensional objects} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method and an apparatus for registering a face, and a method and an apparatus for recognizing a face are disclosed, in which a face registering apparatus may change a stored three-dimensional (3D) facial model to an individualized 3D facial model based on facial landmarks extracted from two-dimensional (2D) face images, match the individualized 3D facial model to a current 2D face image of the 2D face images, and extract an image feature of the current 2D face image from regions in the current 2D face image to which 3D feature points of the individualized 3D facial model are projected, and a face recognizing apparatus may perform facial recognition based on image features of the 2D face images extracted by the face registering apparatus. ## Claims ### Claim 1 A method of recognizing a face, comprising: receiving a two-dimensional (2D) input image; transforming a three-dimensional (3D) facial model based on a user face appearing in the 2D input image; extracting an image feature from positions where feature points of the transformed 3D facial model are projected to the 2D input image; and determining a result of facial recognition based on the extracted image feature and a reference image feature. ### Claim 2 (depends on 1) The method of claim 1, wherein the transforming comprising: adjusting at least one of a facial pose and a facial expression of the 3D facial model based on the user face appearing in the 2D input image. ### Claim 3 (depends on 1) The method of claim 1, wherein the transforming comprising: extracting facial landmarks from the 2D input image; and matching the three-dimensional (3D) facial model to the user face appearing in the 2D input image based on the extracted facial landmarks. ### Claim 4 (depends on 3) The method of claim 3, wherein the extracting comprising: projecting the feature points of the matched 3D facial model to the 2D input image; and extracting the image feature from positions where feature points of the matched 3D facial model are projected to the 2D input image. ### Claim 5 (depends on 1) The method of claim 1, wherein the determining comprising: comparing the extract image feature to a reference image feature; and determining the result of facial recognition based on the comparing. ### Claim 6 (depends on 1) The method of claim 1, wherein the reference image feature comprises an image feature of a 2D face image used for registering a face, obtained by matching the 3D facial model to the 2D face image and extracting the image feature from positions where the feature points of the matched 3D facial model are projected to the 2D face image. ### Claim 7 (depends on 1) The method of claim 1, wherein the determining comprises: selecting a set of reference image features to be used for the facial recognition based on a facial pose in the 2D input image, and determining the result of the facial recognition based on a degree of similarity between the selected set of reference image features and a set of image features extracted from the 2D input image, the reference image feature being in the set of reference image features. ### Claim 8 (depends on 1) The method of claim 1, wherein the determining comprises: determining a degree of similarity between the reference image feature and the extracted image feature; and determining the facial recognition to be successful in response to the degree of similarity satisfying a condition. ### Claim 9 (depends on 1) A non-transitory computer-readable medium comprising program code that, when executed by a processor, performs functions according to the method of claim 1. ### Claim 10 An apparatus for recognizing a face, comprising: at least one processor; and a memory configured to communicate with the at least one processor, the memory configured to store instructions executable by the at least one processor such that the at least one processor is configured to receive a two-dimensional (2D) input image, transform a three-dimensional (3D) facial model based on a user face appearing in the 2D input image, extract an image feature from positions where feature points of the transformed 3D facial model are projected to the 2D input image, and determine a result of facial recognition based on the extracted image feature and a reference image feature. ### Claim 11 (depends on 10) The apparatus of claim 10, wherein the at least one processor is configured to execute the computer-readable instructions to adjust at least one of a facial pose and a facial expression of the 3D facial model based on the user face appearing in the 2D input image. ### Claim 12 (depends on 10) The apparatus of claim 10, wherein the at least one processor is configured to execute the computer-readable instructions to extract facial landmarks from the 2D input image and match the three-dimensional (3D) facial model to the user face appearing in the 2D input image based on the extracted facial landmarks. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein the at least one processor is configured to execute the computer-readable instructions to project the feature points of the matched 3D facial model to the 2D input image and extract the image feature from positions where feature points of the matched 3D facial model are projected to the 2D input image. ### Claim 14 (depends on 10) The apparatus of claim 10, wherein the reference image feature comprises an image feature of a 2D face image used for registering a face, obtained by matching the 3D facial model to the 2D face image and extracting the image feature from positions where the feature points of the matched 3D facial model are projected to the 2D face image. ### Claim 15 A method of registering a face, comprising: receiving a two-dimensional (2D) face image for registering a face; transforming a stored three-dimensional (3D) facial model to an individualized 3D facial model based on a user face appearing in the 2D face image; extracting an image feature from positions where feature points of the individualized 3D facial model are projected to the 2D face image; and storing the extracted image feature as a reference image feature. ### Claim 16 (depends on 15) The method of claim 15, wherein the transforming comprising: extracting facial landmarks from the 2D face image; and matching the stored three-dimensional (3D) facial model to the user face appearing in the 2D face image based on the extracted facial landmarks. ### Claim 17 (depends on 16) The method of claim 16, wherein the extracting comprising: projecting the feature points of the matched 3D facial model to the 2D face image; and extracting the image feature from positions where feature points of the matched 3D facial model are projected to the 2D face image. ### Claim 18 (depends on 16) The method of claim 16, wherein the transforming comprises: determining shape control parameters to match facial landmarks of the stored 3D facial model to the extracted facial landmarks of the 2D face image; and changing the stored 3D facial model to the individualized 3D facial model by applying the determined shape control parameters to the stored 3D facial model. ### Claim 19 (depends on 15) The method of claim 15, wherein the transforming comprising: adjusting at least one of a facial pose and a facial expression of the stored 3D facial model based on the user face appearing in the 2D face image. ### Claim 20 (depends on 15) The method of claim 15, wherein the 2D face image comprise at least one frontal face image including a frontal face of a user and at least one profile image including a profile of the user.