# US 12,223,768 B2 · Adaptive learning and matching of face modalities > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,768 B2 - Document ID: US12223768B2 - Status: Granted - Kind: B2 - Application number: 17/437,764 - Filed: 2020-04-01 - Published: 2022-05-19 - Granted: 2025-02-11 - Earliest claimed: 2019-04-05 - Assignees: RealNetworks, Inc. - Applicants: RealNetworks LLC - Inventors: Milko Boic - Web page: https://patentgrep.com/patent/US12223768B2 ## CPC - G06V 40/167: {using comparisons between temporally consecutive images} - G06F 18/22: Matching criteria, e.g. proximity measures - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06V 10/772: Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries - G06V 10/993: {Evaluation of the quality of the acquired pattern} - G06V 40/165: {using facial parts and geometric relationships} - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06V 40/172: {Classification, e.g. identification} - G06V 40/50: Maintenance of biometric data or enrolment thereof - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Technologies are provided herein for adaptively learning and matching modalities of face images. A face image is obtained and a biometric signature is generated of the face image. The biometric signature is compared with a stored biometric signature obtained from data storage. Differences are determined between the biometric signature and the stored biometric signature and compared with a proximity threshold. As a result of the comparison with the proximity threshold, the identity of the person in the face image may be determined or the face image may be stored in data storage in association with the identity of the person. ## Claims ### Claim 1 A computer-implemented method, comprising: identifying, at a first time, a person in an image based on information associated with an identity stored in data storage; obtaining, at a second time after the first time, a first face image of the person; retrieving, from data storage, a first biometric signature corresponding to the identity; generating a second biometric signature of the first face image; calculating a differential between the first biometric signature and the second biometric signature; comparing the differential to a defined proximity threshold for appearance modalities; obtaining, at a third time after the first time, a second face image of the person; generating a third biometric signature of the second face image; determining an image differential based on a comparison of the second biometric signature and the third biometric signature; and as a result of the differential exceeding the defined proximity threshold and as a result of the image differential being less than an image consistency threshold, storing the second biometric signature in the data storage in association with the identity. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: determining that the first face image has a first appearance modality based on the biometric signature; and determining, as a result of the differential exceeding the defined proximity threshold, that the first biometric signature is of another face image of the person that has a second appearance modality. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the differential corresponds to a distance in multidimensional Euclidean space between the first signature and the second biometric signature, and the defined proximity threshold corresponds to a threshold distance in multidimensional Euclidean space. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising: tracking a face of the person in a sequence of images captured from the first time to the second time, wherein the first face image has a first appearance modality that is different than a second appearance modality of another face image in the sequence of images. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: comparing the differential to a similarity criterion; and as a result of the differential satisfying the similarity criterion, determining that the first face image corresponds to the identity stored in the data storage. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising: generating assessment information for the first face image that includes information regarding one or more characteristics selected from a contrast quality of the first face image, a sharpness of the first face image, a size of the first face image, or an orientation of a face in the first face image; comparing the assessment information for the first face image with a set of criteria; and determining that the assessment information associated with the first face image satisfies the set of criteria, wherein storing the second biometric signature in the data storage in association with the identity is performed as a result of determining that the assessment information satisfies the set of criteria. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, wherein the assessment information for the first face image includes information regarding the contrast quality of the first face image, the sharpness of the first face image, the size of the first face image, and the orientation of a face in the first face image, and the set of criteria includes a contrast quality criterion, a sharpness criterion, a size criterion, and an orientation criterion. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the first biometric signature includes a first multidimensional vector and the second biometric signature includes a second multidimensional vector, and calculating the differential includes determining a difference between the first multidimensional vector and the second multidimensional vector. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein generating the biometric signature comprises processing the first face image using a machine learning model to generate the second biometric signature. ### Claim 10 A system, comprising: at least one processor; and at least one non-transitory storage medium storing a set of instructions that, as a result of execution by the one or more processors, cause the system to: identify, at a first time, a person in an image based on information associated with an identity stored in data storage; obtain, from a camera at a second time after the first time, a first face image of the person; retrieve, from data storage, a first biometric signature corresponding to the identity; generate a second biometric signature of the first face image; calculate a differential between the first biometric signature and the second biometric signature; compare the differential to a defined proximity threshold for appearance modalities; obtaining, at a third time after the first time, a second face image of the person; generating a third biometric signature of the second face image; determining an image differential based on a comparison of the second biometric signature and the third biometric signature; and as a result of the differential exceeding the defined proximity threshold and as a result of the image differential being less than an image consistency threshold, store the second biometric signature in the data storage in association with the identity. ### Claim 11 (depends on 10) The system of claim 10, wherein execution of the set of instructions further causes the system to: determine that the first face image has a first appearance modality based on the biometric signature; and determine, as a result of the differential exceeding the defined proximity threshold, that the first biometric signature is of another face image of the person that has a second appearance modality. ### Claim 12 (depends on 10) The system of claim 10, wherein the differential corresponds to a distance in multidimensional Euclidean space between the first biometric signature and the second biometric signature, and the defined proximity threshold corresponds to a second distance in multidimensional Euclidean space. ### Claim 13 (depends on 10) The system of claim 10, wherein the execution of the set of instructions further causes the system to: track a face of the person in a sequence of images captured from the first time to the second time, wherein the first face image has a first appearance modality that is different than a second appearance modality of another face image in the sequence of images. ### Claim 14 (depends on 10) The system of claim 10, wherein the execution of the set of instructions further causes the system to: compare the differential to a similarity criterion; and as a result of the differential satisfying the similarity criterion, determine that the first face image corresponds to the identity stored in the data storage. ### Claim 15 A non-transitory computer-readable medium storing instructions that, as a result of execution by at least one processor, cause the at least one processor to: identify, at a first time, a person in an image based on information associated with an identity stored in data storage; obtain, at a second time after the first time, a first face image of the person; retrieve, from data storage, a first biometric signature corresponding to the identity; generate a second biometric signature of the first face image; calculate a differential between the first biometric signature and the second biometric signature; compare the differential to a defined proximity threshold for appearance modalities; obtain, at a third time after the first time, a second face image of the person; generate a third biometric signature of the second face image; determine an image differential based on a comparison of the second biometric signature and the third biometric signature; and as a result of the differential exceeding the defined proximity threshold and as a result of the image differential being less than an image consistency threshold, store the second biometric signature in the data storage in association with the identity. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the first biometric signature includes a first multidimensional vector and the second biometric signature includes a second multidimensional vector, and calculating the differential includes determining differences between the first multidimensional vector and the second multidimensional vector. ### Claim 17 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the instructions include a machine learning model that is trained based on a set of biometric signature training data and a corresponding set of differentials calculated for the biometric signature training data.