# US 2019/0318077 A1 · Visual data processing of response images for authentication > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318077 A1 - Document ID: US20190318077A1 - Status: Published application - Kind: A1 - Application number: 16/455,738 - Filed: 2019-06-27 - Published: 2019-10-17 - Earliest claimed: 2016-06-28 - Assignees: Paypal, Inc. - Applicants: Paypal, Inc. - Inventors: Srivathsan Narasimhan - Web page: https://patentgrep.com/patent/US20190318077A1 ## CPC - G06F 21/36: by graphic or iconic representation - G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints - H04L 63/0861: {using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract There are provided systems and methods for visual data processing of mimed images for authentication. Authentication may be required for a user and/or an account of the user, for example, to verify the identity of the user or allow the user to access and use the account of the user. As an additional factor to authentication, increased authentication may be accomplished through who and what the user is, such as through facial recognition and biometrics of the user. During authentication, the user may be presented with a set of images or icons, such as digital emojis, that convey and emotion or idea. The user may be asked to mimic the emojis during authentication, where the user's facial expression is recorded. To authenticate the user, the recorded data may be compared to past data through facial recognition processing and image analysis to find similarities. ## Claims ### Claim 1 A system comprising: a non-transitory memory; and one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising: receiving, over a network connection, an authentication request for a user associated with a mobile device; retrieving, by the system, a plurality of visual data corresponding to the user; generating, by the system, a verification query in response to the authentication request; transmitting, over the network connection, the verification query to a mobile device of the user, the verification query including a request for the user to imitate a particular visual data depicted in an image presented and request for an authentication factor; receiving, over the network connection, a captured visual data depicting a user's imitation of the particular visual data and the authentication factor; and authenticating, by the system, the user, the authenticating based on a comparison between the captured visual data received and the plurality of the visual data retrieved and a verification of the authentication factor. ### Claim 2 (depends on 1) The system of claim 1, wherein the authentication request includes a user login request for the user. ### Claim 3 (depends on 1) The system of claim 1, wherein the plurality of the visual data are retrieved for the user based on a user account information included in the authentication request. ### Claim 4 (depends on 1) The system of claim 1, wherein the particular visual data included in the verification query is an emoji. ### Claim 5 (depends on 1) The system of claim 1, wherein the plurality of visual data include previously captured reactions, pictures, selfies, and/or videos of the user. ### Claim 6 (depends on 1) The system of claim 1, wherein the comparison includes determining a similarity score and exceeding a threshold score. ### Claim 7 (depends on 6) The system of claim 6, executing instructions further cause the system to perform operations comprising: authenticating the user based on the determining of the similarity score and the similarity score exceeding the threshold score. ### Claim 8 (depends on 1) The system of claim 1, wherein the comparison includes determining a similarity score and a level of certainty. ### Claim 9 (depends on 8) The system of claim 8, wherein the authentication factor is verified in response to the level of certainty determined. ### Claim 10 A method comprising: receiving, over a network connection, an authentication request for a user associated with a mobile device; retrieving, by the system, a plurality of visual data corresponding to the user; generating, by the system, a verification query in response to the authentication request; transmitting, over the network connection, the verification query to a mobile device of the user, the verification query including a request for the user to imitate a particular visual data depicted in an image presented and request for an authentication factor; receiving, over the network connection, a captured visual data depicting a user's imitation of the particular visual data and the authentication factor; and authenticating, by the system, the user, the authenticating based on a comparison between the captured visual data received and the plurality of the visual data retrieved and a verification of the authentication factor. ### Claim 11 (depends on 10) The method of claim 10, wherein the plurality of the visual data are retrieved for the user based on a user account information included in the authentication request. ### Claim 12 (depends on 10) The method of claim 10, wherein the particular visual data included in the verification query is an emoji. ### Claim 13 (depends on 10) The method of claim 10, wherein the plurality of visual data include previously captured reactions, pictures, selfies, and/or videos of the user. ### Claim 14 (depends on 10) The method of claim 10, wherein the comparison includes determining a similarity score and exceeding a threshold score. ### Claim 15 (depends on 14) The method of claim 14, further comprising: authenticating the user based on the determining of the similarity score and the similarity score exceeding the threshold score. ### Claim 16 A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising: receiving, over a network connection, an authentication request for a user associated with a mobile device; retrieving, by the system, a plurality of visual data corresponding to the user; generating, by the system, a verification query in response to the authentication request; transmitting, over the network connection, the verification query to a mobile device of the user, the verification query including a request for the user to imitate a particular visual data depicted in an image presented and request for an authentication factor; receiving, over the network connection, a captured visual data depicting a user's imitation of the particular visual data and the authentication factor; and authenticating, by the system, the user, the authenticating based on a comparison between the captured visual data received and the plurality of the visual data retrieved and a verification of the authentication factor. ### Claim 17 (depends on 16) The non-transitory machine-readable medium of claim 16, wherein the comparison includes determining a similarity score and exceeding a threshold score. ### Claim 18 (depends on 17) The non-transitory machine-readable medium of claim 17, executing instructions further cause the system to perform operations comprising: authenticating the user based on the determining of the similarity score and the similarity score exceeding the threshold score. ### Claim 19 (depends on 16) The non-transitory machine-readable medium of claim 16, wherein the comparison includes determining a similarity score and a level of certainty. ### Claim 20 (depends on 19) The non-transitory machine-readable medium of claim 19, wherein the authentication factor is verified in response to the level of certainty determined.