# US 2022/0070236 A1 · Graphical representation-based user authentication system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0070236 A1
- Document ID: US20220070236A1
- Status: Published application
- Kind: A1
- Application number: 17/060,485
- Filed: 2020-10-01
- Published: 2022-03-03
- Earliest claimed: 2020-08-28
- Assignees: TMRW Foundation IP S. À R.L.
- Applicants: Tmrw Foundation IP S. à R.L.
- Inventors: Cevat Yerli
- Web page: https://patentgrep.com/patent/US20220070236A1

## CPC

- H04L 63/0861: {using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231)}
- H04L 65/4053: without floor control
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G06T 19/003: {Navigation within 3D models or images}
- G06V 20/20: in augmented reality scenes
- G06V 20/64: Three-dimensional [3D] objects
- H04L 63/083: {using passwords (cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226)}
- H04L 65/4015: {where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference}
- G06V 40/168: {Feature extraction; Face representation}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/172: {Classification, e.g. identification}
- G06V 40/174: {Facial expression recognition}
- G09B 5/12: different stations being capable of presenting different information simultaneously (G09B5/14 takes precedence)
- H04L 2463/082: applying multi-factor authentication
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

A user authentication system comprising one or more cloud server computers storing a user database storing user data associated with a user account and one or more corresponding user graphical representations, and a facial scanning and authentication module connected to the database. The one or more cloud server computers are configured to authenticate a user by performing a facial scanning of the user, including extracting facial feature data from camera data received from a client device and checking the extracted facial feature data for a match against a user graphical representation associated with the user account in the user database. If a matching user graphical representation is found in the checking step, the system provides the user with access to the corresponding user account. Otherwise, the system generates, from the camera data, a new user graphical representation along with a new user account stored in the user database.

## Claims

### Claim 1

A user authentication system comprising:
one or more cloud server computers comprising at least one processor and memory storing data and instructions comprising a user database storing user data associated with a user account and one or more corresponding user graphical representations, and a facial scanning and authentication module connected to the database;
wherein the one or more cloud server computers are configured to perform steps comprising:
authenticating a user by performing a facial scanning of the user through the facial scanning and authentication module, wherein the facial scanning comprises extracting facial feature data from camera data received from a client device and checking the extracted facial feature data for a match against a user graphical representation associated with the user account in the user database;
if a matching user graphical representation is found in the checking step, providing the user with access to the corresponding user account; and
if a matching user graphical representation is not found in the checking step, generating, from the camera data, a new user graphical representation along with a new user account stored in the user database, and providing access to the new user account.

### Claim 2 (depends on 1)

The system of claim 1, wherein the user graphical representation is a user 3D virtual cutout, or a user real-time 3D virtual cutout with a removed background, or a video with removed background, or a video without removed background.

### Claim 3 (depends on 2)

The system of claim 2, wherein the one or more cloud server computers are further configured to animate the matching user graphical representation or the new user graphical representation, and wherein animating the matching user graphical representation comprises applying machine vision algorithms by the client device or the at least one cloud server computer on the respective user graphical representation for recognizing facial expressions of the user and graphically simulating the facial expressions on the user graphical representation.

### Claim 4 (depends on 1)

The system of claim 1, wherein the one or more cloud server computers are further configured to check a date of the matching user graphical representation and determine whether an update of the matching user graphical representation is required.

### Claim 5 (depends on 1)

The system of claim 1, wherein the user graphical representation is inserted into a two-or-three dimensional virtual environment or on a third-party source linked to a virtual environment.

### Claim 6 (depends on 1)

The system of claim 1, wherein the generation process of the new user graphical representation takes place asynchronously from user access to the user account.

### Claim 7 (depends on 1)

The system of claim 1, wherein the one or more cloud server computers are further configured to authenticate the user through login authentication credentials comprising a personal identification number (PIN), or username and password, or a combination thereof.

### Claim 8 (depends on 1)

The system of claim 1, wherein the authenticating is triggered in response to activation of an invitation link sent from by one client device to another client device.

### Claim 9 (depends on 1)

The system of claim 1, wherein the facial scanning uses 3D authentication comprising guiding a user to perform a head movement pattern and extracting 3D face data based on the head movement pattern.

### Claim 10 (depends on 9)

The system of claim 9, wherein the 3D authentication uses one or more deep learning techniques to authenticate the user using the 3D face data.

### Claim 11

A user authentication method comprising:
providing in memory of one or more cloud server computers a user database storing user data associated with a user account and one or more corresponding user graphical representations, and a facial scanning and authentication module connected to the database;
receiving, from a client device, a request to access the user account;
performing a facial scanning of a user of the client device through the facial scanning and authentication module by extracting facial feature data from camera data captured by at least one camera in communication with the client device;
checking the extracted facial feature data for a match against a user graphical representation associated with the user account in the user database;
if a matching user graphical representation is found in the checking step, providing the user with access to the user account; and
if a matching user graphical representation is not found in the checking step, generating, from the camera data, a new user graphical representation along with a new user account stored in the user database, and providing access to the new user account.

### Claim 12 (depends on 11)

The method of claim 11, wherein the user graphical representation is a user 3D virtual cutout, or a user real-time 3D virtual cutout, or a video with removed background, or video without removed background.

### Claim 13 (depends on 12)

The method of claim 12, wherein the one or more cloud server computers are further configured to animate the matching user graphical representation or the new user graphical representation, and wherein animating the matching user graphical representation comprises applying machine vision algorithms by the client device or the at least one cloud server computer on a generated user 3D virtual cutout for recognizing facial expressions of the user and graphically simulating the facial expressions on the user 3D virtual cutout.

### Claim 14 (depends on 11)

The method of claim 11, further comprising, if a matching user graphical representation is found in the checking step:
checking a date of the matching user graphical representation;
determining, based at least in part on the date, whether an update of the matching user graphical representation is required by comparing to corresponding threshold values or security requirements; and
in a positive case where an update of the matching user graphical representation is required, generating a user graphical representation update request.

### Claim 15 (depends on 11)

The method of claim 11, further comprising inserting the user graphical representation into a two-or-three dimensional virtual environment or on a third-party source linked to a virtual environment.

### Claim 16 (depends on 11)

The method of claim 11, wherein the generation process of the new user graphical representation takes place asynchronously from user access to the user account.

### Claim 17 (depends on 11)

The method of claim 11, further comprising authenticating the user through login authentication credentials comprising username and password.

### Claim 18 (depends on 11)

The method of claim 11, wherein the authenticating is triggered in response to activation of an invitation link.

### Claim 19

A non-transitory computer readable medium having stored thereon instructions configured to cause at least one server computer comprising a processor and memory to perform steps comprising:
providing in memory of one or more cloud server computers a user database storing user data associated with a user account and one or more corresponding user graphical representations, and a facial scanning and authentication module connected to the database;
receiving, from a client device, a request to access the user account;
performing a facial scanning of a user of the client device through the facial scanning and authentication module by extracting facial feature data from camera data captured by at least one camera in communication with the client device;
checking the extracted facial feature data for a match against a user graphical representation associated with the user account in the user database;
if a matching user graphical representation is found in the checking step, providing the user with access to the user account; and,
if a matching user graphical representation is not found in the checking step, generating, from the camera data, a new user graphical representation along with a new user account stored in the user database, and providing access to the new user account.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, further comprising, if a matching user graphical representation is found in the checking step:
checking a date of the matching user graphical representation:
determining, based at least in part on the date, whether an update of the matching user graphical representation is required; and
in a positive case where an update of the matching user graphical representation is required, generating a user graphical representation update request.
