# US 12,183,117 B2 · User authentication using pose-based facial recognition > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,117 B2 - Document ID: US12183117B2 - Status: Granted - Kind: B2 - Application number: 17/433,912 - Filed: 2019-04-03 - Published: 2022-05-05 - Granted: 2024-12-31 - Earliest claimed: 2019-02-28 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Cem Kemal Hamami; Joseph Edwin Johnson, Jr.; Kuntal Sengupta; Piotr Kulaga; Wen-Sheng Chu; Zachary Iqbal - Web page: https://patentgrep.com/patent/US12183117B2 ## CPC - G06V 40/172: {Classification, e.g. identification} - G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints - G06F 21/44: Program or device authentication - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/772: Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries - G06V 40/173: {face re-identification, e.g. recognising unknown faces across different face tracks} - G06V 40/50: Maintenance of biometric data or enrolment thereof - G06F 2221/2141: Access rights, e.g. capability lists, access control lists, access tables, access matrices - G06T 2207/30201: Face - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method includes receiving data indicative of an image of a face of an unknown user of the computing device while the computing device is in a reduced access mode locked state. The method also includes determining whether the unknown user is the known user by at least comparing the image of the face of the unknown user to one or more images of a plurality of images of a face of a known user of the computing device. The method further includes setting the computing device to an increased access mode in response to determining that the unknown user is the known user. ## Claims ### Claim 1 A method comprising: performing, by at least one processor of a computing device, an enrollment process for a known user by at least: receiving, by the at least one processor, data indicative of a plurality of images of a face of the known user; determining, by the at least one processor, a number of pose buckets of a plurality of pose buckets that include the plurality of images of the face of the known user, wherein: each of the plurality of images of the face of the known user is included in at least one pose bucket from a plurality of pose buckets, and each pose bucket from the plurality of pose buckets is associated with a respective range of pitch angles of the face of the known user and a respective range of yaw angles of the face of the known user; determining, by the at least one processor, whether the number of pose buckets satisfies a threshold number of pose buckets; and responsive to determining that the number of pose buckets satisfies a threshold number of pose buckets, associating, by the at least one processor, the data indicative of the plurality of images with a user account for the known user; subsequent to performing the enrollment process and while the computing device is in a reduced access mode: receiving, by the at least one processor, data indicative of an image of a face of an unknown user of the computing device; and determining, by the at least one processor, whether the unknown user is the known user of the computing device by at least comparing the data indicative of the image of the face of the unknown user to data indicative of one or more images of the plurality of images of the face of the known user; and responsive to determining that the unknown user is the known user, setting, by the at least one processor, the computing device to an increased access mode. ### Claim 2 (depends on 1) The method of claim 1, further comprising: selecting, by the at least one processor, from the plurality of images of the face of the known user, an image included in a particular pose bucket of the plurality of pose buckets, wherein the particular pose bucket is associated with the image of the face of the unknown user, and wherein determining whether the unknown user is the known user is based on the image of the face of the unknown user and the selected image of the face of the known user. ### Claim 3 (depends on 2) The method of claim 2, further comprising: determining, by the at least one processor, a similarity score indicating a similarity between the image of the face of the unknown user and the selected image of the face of the known user; and determining, by the computing device, the unknown user is the known user in response to determining that the similarity score satisfies a threshold similarity score. ### Claim 4 (depends on 1) The method of claim 1, further comprising: determining, by the at least one processor, based on the image of the face of the unknown user and each image of the plurality of images of the face of the known user, a respective similarity score for each image of the plurality of images of the face of the known user, wherein each similarity score indicates a similarity between the image of the face of the unknown user and the respective image of the plurality of images of the face of the known user, wherein determining whether the unknown user is the known user is based on the respective similarity score for the one or more images of the plurality of images of the face of the known user. ### Claim 5 (depends on 4) The method of claim 4, further comprising: determining, by the at least one processor, based on the respective similarity scores, a highest ranked image of the plurality of images of the face of the known user, wherein determining whether the unknown user is the known user is based on the highest ranked image of the plurality of images of the face of the known user. ### Claim 6 (depends on 4) The method of claim 4, further comprising: determining, by the at least one processor, based on the similarity scores for two or more images of the plurality of images of the face of the known user, a composite similarity score, wherein determining whether the unknown user is the known user comprises determining, by the at least one processor, the unknown user is the known user in response to determining that the composite similarity score satisfies a threshold similarity score. ### Claim 7 (depends on 4) The method of claim 4, wherein determining whether the unknown user is the known user comprises determining, by the at least one processor, the unknown user is the known user in response to determining that the respective similarity score for each of two or more images of the face of the known user satisfies a threshold similarity score. ### Claim 8 (depends on 1) The method of claim 1, further comprising: receiving, by the at least one processor, a first image of the plurality of images, wherein a pitch angle and a yaw angle of the first image lie within a range of pitch angles and yaw angles associated with a particular pose bucket; receiving, by the at least one processor, a second image of the plurality of images, wherein a pitch angle and a yaw angle of the second image lie within the range of pitch angles and yaw angles associated with the particular pose bucket; and determining, by the at least one processor, whether to include the first image or the second image in the particular pose bucket. ### Claim 9 (depends on 8) The method of claim 8, wherein determining whether to include the first image or the second image in the particular pose bucket includes: determining, by the at least one processor, a center of the particular pose bucket; determining, by the at least one processor, based on the pitch angle and the yaw angle of the first image and the pitch angle and the yaw angle of the second image, that the second image is closer to the center of the particular pose bucket; and determining to include the second image in the particular pose bucket in response to determining that the second image is closer to the center of the particular pose bucket. ### Claim 10 (depends on 1) The method of claim 1, wherein receiving the data indicative of the plurality of images of the face of the known user comprises receiving, by the at least one processor, from a plurality of cameras, data indicative of 3-dimensional images of the face of the known user. ### Claim 11 (depends on 1) The method of claim 1, wherein receiving the data indicative of the image of the face of the unknown user comprises receiving data indicative of a 2-dimensional image and data indicative of a 3-dimensional image of the face of the unknown user, wherein the plurality of images of the face of the known user includes a plurality of 2-dimensional images of images of the face of the known user and a plurality of 3-dimensional images of images of the face of the known user, and determining whether the unknown user is the known user is based on the 2-dimensional image of the face of the unknown user, the 3-dimensional image of the face of the unknown user, one or more 2-dimensional images of the plurality of 2-dimensional images of the face of the known user, and one or more 3-dimensional images of the plurality of 3-dimensional images of the face of the known user. ### Claim 12 A computing device comprising: a memory that stores instructions; and one or more processors to: perform an enrollment process for a known user of a computing device by causing the one or more processors to at least: receive data indicative of a plurality of images of a face of the known user; determine a number of pose buckets of a plurality of pose buckets that include the plurality of images of the face of the known user, wherein: each of the plurality of images of the face of the known user is included in at least one pose bucket from a plurality of pose buckets, and each pose bucket from the plurality of pose buckets is associated with a respective range of pitch angles of the face of the known user and a respective range of yaw angles of the face of the known user; determine whether the number of pose buckets satisfies a threshold number of pose buckets; and responsive to determining that the number of pose buckets satisfies a threshold number of pose buckets, associate the data indicative of the plurality of images with a user account for the known user; subsequent to performing the enrollment process and while the computing device is in a reduced access mode: receive data indicative of an image of a face of an unknown user of the computing device; and determine whether the unknown user is the known user of the computing device by at least comparing the data indicative of the image of the face of the unknown user to data indicative of one or more images of the plurality of images of the face of the known user; and responsive to determining that the unknown user is the known user, set the computing device to an increased access mode. ### Claim 13 (depends on 12) The computing device of claim 12, wherein the one or more processors further execute the instructions to: select, from the plurality of images of the face of the known user, an image included in a particular pose bucket of the plurality of pose buckets, wherein the particular pose bucket is associated with the image of the face of the unknown user, and wherein the instructions that cause the one or more processors to determine whether the unknown user is the known user further cause the one or more processors to determine whether the unknown user is the known user based on the image of the face of the unknown user and the selected image of the face of the known user. ### Claim 14 (depends on 12) The computing device of claim 12, wherein the one or more processors further execute the instructions to: determine, based on the image of the face of the unknown user and each image of the plurality of images of the face of the known user, a respective similarity score for each image of the plurality of images of the face of the known user, wherein each similarity score indicates a similarity between the image of the face of the unknown user and the respective image of the plurality of images of the face of the known user, wherein the instructions that cause the one or more processors to determine whether the unknown user is the known user further cause the one or more processors to determine whether the unknown user is the known user based on the respective similarity score for the one or more images of the plurality of images of the face of the known user. ### Claim 15 A non-transitory computer-readable storage medium encoded with instructions that, when executed by one or more processors of a computing device, cause the one or more processors to: perform an enrollment process for a known user of the computing device by causing the one or more processors to at least: receive data indicative of a plurality of images of a face of the known user; determine a number of pose buckets of a plurality of pose buckets that include the plurality of images of the face of the known user, wherein: each of the plurality of images of the face of the known user is included in at least one pose bucket from a plurality of pose buckets, and each pose bucket from the plurality of pose buckets is associated with a respective range of pitch angles of the face of the known user and a respective range of yaw angles of the face of the known user; determine whether the number of pose buckets satisfies a threshold number of pose buckets; and responsive to determining that the number of pose buckets satisfies a threshold number of pose buckets, associate the data indicative of the plurality of images with a user account for the known user; subsequent to performing the enrollment process and while the computing device is in a reduced access mode: receive data indicative of an image of a face of an unknown user of the computing device; and determine whether the unknown user is the known user of the computing device by at least comparing the data indicative of the image of the face of the unknown user to data indicative of one or more images of the plurality of images of the face of the known user; and responsive to determining that the unknown user is the known user, set the computing device to an increased access mode. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: select, from the plurality of images of the face of the known user, an image included in a particular pose bucket of the plurality of pose buckets, wherein the particular pose bucket is associated with the image of the face of the unknown user, and wherein the instructions that cause the one or more processors to determine whether the unknown user is the known user further cause the one or more processors to determine whether the unknown user is the known user based on the image of the face of the unknown user and the selected image of the face of the known user. ### Claim 17 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine, based on the image of the face of the unknown user and each image of the plurality of images of the face of the known user, a respective similarity score for each image of the plurality of images of the face of the known user, wherein each similarity score indicates a similarity between the image of the face of the unknown user and the respective image of the plurality of images of the face of the known user, wherein the instructions that cause the one or more processors to determine whether the unknown user is the known user further cause the one or more processors to determine whether the unknown user is the known user based on the respective similarity score for the one or more images of the plurality of images of the face of the known user.