# US 2021/0176239 A1 · Facial Recognition, Image Analysis, and Decentralized Learning Framework Using Adaptive Security Protocols in Distributed Terminal Network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0176239 A1
- Document ID: US20210176239A1
- Status: Published application
- Kind: A1
- Application number: 16/811,320
- Filed: 2020-03-06
- Published: 2021-06-10
- Earliest claimed: 2019-12-09
- Inventors: Evan Chase Rose
- Web page: https://patentgrep.com/patent/US20210176239A1

## CPC

- H04L 63/0861: {using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231)}
- G06F 21/45: Structures or tools for the administration of authentication
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 20/00: Machine learning
- G06Q 30/0185: {Product, service or business identity fraud}
- H04L 9/3226: {using a predetermined code, e.g. password, passphrase or PIN (network architectures or network communication protocols for supporting authentication of entities using passwords in a packet data network H04L63/083)}
- H04L 12/4641: {Virtual LANs, VLANs, e.g. virtual private networks [VPN] (LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272)}
- H04L 63/0272: {Virtual private networks}
- H04L 63/0428: {wherein the data content is protected, e.g. by encrypting or encapsulating the payload}
- H04L 63/08: {for authentication of entities (cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32)}
- H04L 63/0823: {using certificates (cryptographic mechanisms or cryptographic arrangements for entity authentication involving certificates H04L9/3263)}
- H04L 63/102: {Entity profiles}
- H04L 63/1416: {Event detection, e.g. attack signature detection}
- H04L 63/145: {the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms}
- H04L 63/166: {at the transport layer}
- H04L 63/168: {above the transport layer}
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04W 12/72: Subscriber identity
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Biometric authentication, decentralizd learning frameworks, and adaptive security protocols and services for a network of distributed terminals are described. In some embodiments, the terminals may be hardware terminals, kiosks, or clients. In some embodiments, a security analysis may be performed, and security scores may be determined, for visitors requesting operations at terminals. Security scores may be determined by a vendor, in communication with the operator terminals, based on aggregation of a plurality of factors, wherein each factor may be weighted. The factors may incorporate operator settings or preferences. In one embodiment, the factors include one or more facial recognition factors. The one or more facial recognition factors may be used for biometric authentication. The vendor may use the security scores to determine user privileges or permissions for the operations. The vendor may deliver instructions or messages to the terminals based on the determinations.

## Claims

### Claim 1

A method for biometric authentication and securely handling an operation between a first visitor and an operator of a client, the method comprising:
downloading or storing locally to the client, in at least one computer memory, a machine learning model or machine learning model parameters, wherein the client comprises a hardware terminal or edge server;
wherein the hardware terminal is one of a dispensary terminal, a client terminal, a kiosk, a node, or a vending machine;
wherein the machine learning model comprises, at least:
a trained or pre-trained deep neural network or convolutional neural network based model;
wherein the deep neural network or convolutional neural network based model has been trained on, at least:
a dataset including information about known customers and/or restricted users;
wherein the dataset includes data based on images of the known customers and restricted users, the images including first facial features of the known customers and/or restricted users;
receiving an operation from a first visitor at the client;
identifying local facial recognition data associated with the first visitor, wherein the identifying the local facial recognition data associated with the first visitor comprises:
gathering, using a camera, image or video data of the first visitor's face;
extracting a plurality of second facial features associated with the image or video data;
calculating, at the client, using the machine learning model whether the first visitor matches a known user or restricted user, wherein the calculating comprises, at least:
determining a nearest neighbor probability;
determining whether the nearest neighbor probability exceeds a threshold probability;
based on the calculating, determining whether to authenticate the first visitor,
updating, at the client, the machine learning model using the local facial recognition data to produce a first updated machine learning model;
establishing a secure session between a software service provider and the client;
wherein the secure session is established using at least a secure socket layer (SSL) or transport layer security (TLS) protocol;
wherein the secure session is established using a secured connection using a virtual private network (VPN);
performing a first processing of the operation, wherein performing the first processing of the operation comprises:
sending, by the client, an encrypted first payload;
wherein the encrypted first payload is produced by encrypting a first payload, the first payload produced by the client, and wherein the first payload comprises first data comprising a phone number, the phone number received from the first visitor at the client;
wherein the encrypted first payload is communicated to the software service provider from the client during the secure session using the secured connection;
performing a second processing of the operation, wherein performing the second processing of the operation comprises:
sending, by the client, an encrypted second payload;
wherein the encrypted second payload is produced by encrypting a second payload, the second payload produced by the client;
wherein the encrypted second payload is communicated to the software service provider from the client during the secure session using the secured connection;
performing a third processing of the operation, wherein performing the third processing of the operation comprises:
sending, by the client, an encrypted third payload;
wherein the encrypted third payload is produced by encrypting a third payload, the third payload produced by the client, and wherein the third payload comprises at least second data;
wherein the encrypted third payload is communicated to the software service provider from the client during the secure session using the secured connection;
identifying a facial recognition factor associated with the first visitor, wherein the identifying the facial recognition factor associated with the first visitor comprises:
sending, by the client, an encrypted fourth payload;
wherein the encrypted fourth payload is produced by encrypting a fourth payload, the fourth payload produced by the client, and wherein the fourth payload comprises at least one parameter associated with image or video data associated with the first customer's face;
wherein the encrypted fourth payload is communicated to the software service provider from the client during the secure session using the secured connection.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
wherein the first payload is a JSON encoded object.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
wherein the second payload is a JSON encoded object.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
wherein the third payload is a JSON encoded object.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
wherein the fourth payload is a JSON encoded object.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
downloading or storing locally to the client, in the least one computer memory, updated machine learning model parameters reflecting a second updated machine learning model;
wherein the second updated machine learning model is a global model that includes, at least, updates based on the local facial recognition data.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
referring to the first updated machine learning model or second updated machine learning model for a subsequent visitor authentication request.

### Claim 8

A system comprising:
one or more web servers comprising:
one or more processors;
one or more databases or datastores;
the system to execute a method for biometric authentication and securely handling an operation between a first visitor and an operator of a client, the method comprising:
downloading or storing locally to the client, in at least one computer memory, a machine learning model or machine learning model parameters, wherein the client comprises a hardware terminal or edge server;
wherein the hardware terminal is one of a dispensary terminal, a client terminal, a kiosk, a node, or a vending machine;
wherein the machine learning model comprises, at least:
a trained or pre-trained deep neural network or convolutional neural network based model;
wherein the deep neural network or convolutional neural network based model has been trained on, at least:
a dataset including information about known customers and/or restricted users;
wherein the dataset includes data based on images of the known customers and restricted users, the images including first facial features of the known customers and/or restricted users;
receiving an operation from a first visitor at the client;
identifying local facial recognition data associated with the first visitor, wherein the identifying the local facial recognition data associated with the first visitor comprises:
gathering, using a camera, image or video data of the first visitor's face;
extracting a plurality of second facial features associated with the image or video data;
calculating, at the client, using the machine learning model whether the first visitor matches a known user or restricted user, wherein the calculating comprises, at least:
determining a nearest neighbor probability;
determining whether the nearest neighbor probability exceeds a threshold probability;
based on the calculating, determining whether to authenticate the first visitor,
updating, at the client, the machine learning model using the local facial recognition data to produce a first updated machine learning model;
establishing a secure session between a software service provider and the client;
wherein the secure session is established using at least a secure socket layer (SSL) or transport layer security CLS) protocol;
wherein the secure session is established using a secured connection using a virtual private network (VPN);
performing a first processing of the operation, wherein performing the first processing of the operation comprises:
sending by the client, an encrypted first payload;
wherein the encrypted first payload is produced by encrypting a first payload, the first payload produced by the client, and wherein the first payload comprises first data comprising a phone number, the phone number received from the first visitor at the client;
wherein the encrypted first payload is communicated to the software service provider from the client during the secure session using the secured connection;
performing a second processing of the operation, wherein performing the second processing of the operation comprises:
sending, by the client, an encrypted second payload;
wherein the encrypted second payload is produced by encrypting a second payload, the second payload produced by the client;
wherein the encrypted second payload is communicated to the software service provider from the client during the secure session using the secured connection;
performing a third processing of the operation, wherein performing the third processing of the operation comprises:
sending, by the client, an encrypted third payload;
wherein the encrypted third payload is produced by encrypting a third payload, the third payload produced by the client, and wherein the third payload comprises at least second data;
wherein the encrypted third payload is communicated to the software service provider from the client during the secure session using the secured connection;
identifying a facial recognition factor associated with the first visitor, wherein the identifying the facial recognition factor associated with the first visitor comprises:
sending, by the client, an encrypted fourth payload;
wherein the encrypted fourth payload is produced by encrypting a fourth payload, the fourth payload produced by the client, and wherein the fourth payload comprises at least one parameter associated with image or video data associated with the first customer's face;
wherein the encrypted fourth payload is communicated to the software service provider from the client during the secure session using the secured connection.

### Claim 9 (depends on 8)

The system of claim 8, further comprising:
wherein the first payload is a JSON encoded object.

### Claim 10 (depends on 9)

The system of claim 9, further comprising:
wherein the second payload is a JSON encoded object.

### Claim 11 (depends on 10)

The system of claim 10, further comprising:
wherein the third payload is a JSON encoded object.

### Claim 12 (depends on 11)

The system of claim 11, further comprising:
wherein the fourth payload is a JSON encoded object.

### Claim 13 (depends on 8)

The system of claim 8, further comprising:
downloading or storing locally to the client, in the least one computer memory, updated machine learning model parameters reflecting a second updated machine learning model;
wherein the second updated machine learning model is a global model that includes, at least, updates based on the local facial recognition data.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
referring to the first updated machine learning model or second updated machine learning model for a subsequent visitor authentication request.

### Claim 15

One or more non-transitory computer readable media storing instructions that, when executed, cause one or more processors to perform a method comprising:
downloading or storing locally to the client, in at least one computer memory, a machine learning model or machine learning model parameters, wherein the client comprises a hardware terminal or edge server;
wherein the hardware terminal is one of a dispensary terminal, a client terminal, a kiosk, a node, or a vending machine;
wherein the machine learning model comprises, at least:
a trained or pre-trained deep neural network or convolutional neural network based model;
wherein the deep neural network or convolutional neural network based model has been trained on, at least:
a dataset including information about known customers and/or restricted users;
wherein the dataset includes data based on images of the known customers and restricted users, the images including first facial features of the known customers and/or restricted users;
receiving an operation from a first visitor at the client;
identifying local facial recognition data associated with the first visitor, wherein the identifying the local facial recognition data associated with the first visitor comprises:
gathering, using a camera, image or video data of the first visitor's face;
extracting a plurality of second facial features associated with the image or video data;
calculating, at the client, using the machine learning model whether the first visitor matches a known user or restricted user, wherein the calculating comprises, at least:
determining a nearest neighbor probability;
determining whether the nearest neighbor probability exceeds a threshold probability;
based on the calculating, determining whether to authenticate the first visitor;
updating, at the client, the machine learning model using the local facial recognition data to produce a first updated machine learning model;
establishing a secure session between a software service provider and the client;
wherein the secure session is established using at least a secure socket layer (SSL) or transport layer security CLS) protocol;
wherein the secure session is established using a secured connection using a virtual private network (VPN);
performing a first processing of the operation, wherein performing the first processing of the operation comprises:
sending, by the client, an encrypted first payload;
wherein the encrypted first payload is produced by encrypting a first payload, the first payload produced by the client, and wherein the first payload comprises first data comprising a phone number, the phone number received from the first visitor at the client;
wherein the encrypted first payload is communicated to the software service provider from the client during the secure session using the secured connection;
performing a second processing of the operation, wherein performing the second processing of the operation comprises:
sending, by the client, an encrypted second payload;
wherein the encrypted second payload is produced by encrypting a second payload, the second payload produced by the client;
wherein the encrypted second payload is communicated to the software service provider from the client during the secure session using the secured connection;
performing a third processing of the operation, wherein performing the third processing of the operation comprises:
sending, by the client, an encrypted third payload;
wherein the encrypted third payload is produced by encrypting a third payload, the third payload produced by the client, and wherein the third payload comprises at least second data;
wherein the encrypted third payload is communicated to the software service provider from the client during the secure session using the secured connection;
identifying a facial recognition factor associated with the first visitor, wherein the identifying the facial recognition factor associated with the first visitor comprises:
sending, by the client, an encrypted fourth payload;
wherein the encrypted fourth payload is produced by encrypting a fourth payload, the fourth payload produced by the client, and wherein the fourth payload comprises at least one parameter associated with image or video data associated with the first customer's face;
wherein the encrypted fourth payload is communicated to the software service provider from the client during the secure session using the secured connection.

### Claim 16 (depends on 15)

The one or more non-transitory computer readable media of claim 15, further comprising:
wherein the first payload is a JSON encoded object.

### Claim 17 (depends on 16)

The one or more non-transitory computer readable media of claim 16, further comprising:
wherein the second payload is a JSON encoded object.

### Claim 18 (depends on 17)

The one or more non-transitory computer readable media of claim 17, further comprising:
wherein the third payload is a JSON encoded object.

### Claim 19 (depends on 18)

The one or more non-transitory computer readable media of claim 18, further comprising:
wherein the fourth payload is a JSON encoded object.

### Claim 20 (depends on 19)

The one or more non-transitory computer readable media of claim 19, further comprising:
downloading or storing locally to the client, in the least one computer memory, updated machine learning model parameters reflecting a second updated machine learning model;
wherein the second updated machine learning model is a global model that includes, at least, updates based on the local facial recognition data; and
referring to the first updated machine learning model or second updated machine learning model for a subsequent visitor authentication request.
