# US 12,118,560 B2 · Pointer movement modelling for entity classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,560 B2
- Document ID: US12118560B2
- Status: Granted
- Kind: B2
- Application number: 17/651,081
- Filed: 2022-02-15
- Published: 2023-08-17
- Granted: 2024-10-15
- Earliest claimed: 2022-02-15
- Assignees: Paypal, Inc.
- Applicants: PayPal, Inc.
- Inventors: Zhe Chen; Hewen Wang; Solomon Kok How Teo; Yuzhen Zhuo; Quan Jin Ferdinand Tang; Mandar Ganaba Gaonkar; Omkumar Mahalingam; Kenneth Bradley Snyder
- Web page: https://patentgrep.com/patent/US12118560B2

## CPC

- G06Q 20/4016: {involving fraud or risk level assessment in transaction processing}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 21/31: User authentication
- G06N 20/20: Ensemble learning
- G06Q 20/40145: {Biometric identity checks}
- G06V 10/82: using neural networks
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06V 40/40: Spoof detection, e.g. liveness detection
- G06F 2221/2133: Verifying human interaction, e.g., Captcha
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Techniques are disclosed relating to determining whether to authorize a requested action based on whether an entity is an automated computer. In some embodiments, a computer system tracks, at a user interface of a computing device, a sequence of pointer movements. The computer system maps, based on coordinate locations of pointer movements in the sequence, respective movements in the sequence to a plurality of functional areas. Based on the mapping, the computer system generates a movement graph and determines, based on the movement graph, whether an entity associated with the sequence of pointer movements is an automated computer. In response to receiving a request to authorize an action at the computing device, the computer system generates, based on the determining, an authorization decision for the action and transmits the authorization decision to the computing device. Determining whether the entity is an automated computer may advantageously prevent fraudulent activity.

## Claims

### Claim 1

A method, comprising:
tracking, by a server computer system based on pointer movement information identified at a mobile application rendered at a user interface of a mobile device, a sequence of pointer movements, wherein the pointer movement information is transmitted from the mobile device to the server computer system;
mapping, by the server computer system based on coordinate locations of pointer movements in the sequence of pointer movements, respective movements in the sequence to a plurality of functional areas;
generating, by the server computer system based on the mapping, a movement graph, wherein the generating includes:
assigning graph nodes to respective functional areas included in the plurality of functional areas; and
assigning graph edges to respective pointer movements in the sequence of pointer movements between respective functional areas included in the plurality of functional areas, wherein weights assigned to graph edges are based on a number of pointer movements between pairs of functional areas;
determining, by the server computer system based on the movement graph, whether an entity associated with the sequence of pointer movements is an automated computer;
in response to receiving a request to authorize an action at the mobile device, generating, by the server computer system based on the determining, an authorization decision for the action; and
transmitting, by the server computer system to the mobile device, the authorization decision.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
inputting, by the server computer system, the movement graph into a first machine learning model;
generating, by the server computer system based on output of the first machine learning model, a representative set of pointer movement data; and
inputting, by the server computer system, the representative set of pointer movement data into a second, different machine learning model, wherein the determining whether the entity is an automated computer is further based on output of the second, different machine learning model.

### Claim 3 (depends on 2)

The method of claim 2, wherein the second, different machine learning model is generated by:
generating, from a plurality of historical sequences of pointer movements tracked via user interfaces of a plurality of computing devices, a plurality of representative sets of movement data;
inputting the plurality of representative sets of movement data into the second, different machine learning model; and
adjusting, based on output of the second, different machine learning model for the representative sets of movement data and known labels for the historical sequences of pointer movements corresponding to the representative sets of movement data, one or more weights of the second, different machine learning model.

### Claim 4 (depends on 2)

The method of claim 2, wherein the first machine learning model is a graph convolutional network (GCN), and wherein the GCN is trained by:
generating, from a plurality of historical sequences of pointer movements tracked via user interfaces of a plurality of computing devices, a historical movement graph; and
inputting the historical movement graph into the GCN, wherein the GCN includes at least two neural network layers.

### Claim 5 (depends on 2)

The method of claim 2, further comprising:
generating, by the server computer system based on the sequence of pointer movements, a set of pointer biometric data.

### Claim 6 (depends on 5)

The method of claim 5, wherein output of the first machine learning model for the movement graph is a set of regression classifiers, and wherein generating the representative set of movement data includes:
performing a dot product of the set of regression classifiers and the sequence of pointer movements.

### Claim 7 (depends on 5)

The method of claim 5, wherein generating the set of pointer biometric data includes calculating one or more of the following biometric features for respective pointer movements included in the sequence of pointer movements: movement speed, movement acceleration, movement directions, movement intervals, and movement distance.

### Claim 8 (depends on 1)

The method of claim 1, wherein the sequence of pointer movements are consecutive pointer movements within the user interface of the mobile device, and wherein the map of the sequence of pointer movements corresponds to a pointer trajectory.

### Claim 9 (depends on 1)

The method of claim 1, wherein one or more of the plurality of functional areas include a point of interest within the user interface.

### Claim 10

A non-transitory computer-readable medium having instructions stored thereon that are executable by a server computer system to perform operations comprising:
tracking, based on pointer movement information identified at a browser rendered at a user interface of a desktop computer, a sequence of pointer movements, wherein the pointer movement information is transmitted from the desktop computer to the server computer system;
mapping, based on coordinate locations of pointer movements in the sequence of pointer movements, respective movements in the sequence to a plurality of functional areas;
generating a movement graph based on the mapping, wherein the generating includes:
assigning graph nodes to respective functional areas included in the plurality of functional areas; and
assigning graph edges to respective pointer movements in the sequence of pointer movements between respective functional areas included in the plurality of functional areas;
determining, based on output of the movement graph, whether an entity associated with the sequence of pointer movements is an automated computer;
in response to receiving a request to authorize an action at the desktop computer, generating an authorization decision for the action based on the determining; and
transmitting, to the desktop computer, the authorization decision.

### Claim 11 (depends on 10)

The non-transitory computer-readable medium of claim 10, wherein the operations further comprise:
inputting the movement graph into a first machine learning model;
generating, based on output of the first machine learning model, a representative set of pointer movement data; and
inputting the representative set of pointer movement data into a second, different machine learning model, wherein the determining whether the entity is an automated computer is further based on output of the second, different machine learning model.

### Claim 12 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein the first machine learning model is a graph convolution network (GCN), and wherein the GCN is trained by:
generating, from a plurality of historical sequences of pointer movements tracked via user interfaces of a plurality of computing devices, a historical movement graph; and
inputting the historical movement graph into the GCN, wherein the GCN includes at least three neural network layers.

### Claim 13 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein the operations further comprise:
generating, based on the sequence of pointer movements, a set of pointer biometric data.

### Claim 14 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein output of the first machine learning model for the movement graph is a set of regression classifiers, and wherein generating the representative set of movement data includes:
performing a dot product of the set of regression classifiers and the sequence of pointer movements.

### Claim 15 (depends on 10)

The non-transitory computer-readable medium of claim 10,
wherein weights assigned to graph edges are based on a number of pointer movements between pairs of functional areas.

### Claim 16

A system, comprising:
at least one processor; and
a memory having instructions stored thereon that are executable by the at least one processor to cause the system to:
track, at a user interface of a mobile device, a sequence of pointer movements, wherein the tracking is based on pointer movement information determined at a mobile application rendered via the user interface of the mobile device and transmitted from the mobile device to the system;
generate, based on the sequence of pointer movements, a movement graph, wherein the generating includes:
mapping, based on coordinate locations of pointer movements in the sequence of pointer movements, respective movements in the sequence to a plurality of functional areas;
assigning graph nodes to respective functional areas included in the plurality of functional areas; and
assigning graph edges to respective pointer movements in the sequence of pointer movements between respective functional areas included in the plurality of functional areas, wherein weights assigned to graph edges are based on a number of pointer movements between pairs of functional areas;
input the movement graph into a first machine learning model;
generate, based on output of the first machine learning model, a representative set of pointer movement data;
input the representative set of pointer movement data into a second, different machine learning model;
determine, based on output of the second, different machine learning model, whether an entity associated with the sequence of pointer movements is an automated computer;
in response to receiving a request to authorize an action at the mobile device, generate, based on the determining, an authorization decision for the action; and
transmit, to the mobile device, the authorization decision.

### Claim 17 (depends on 16)

The system of claim 16, wherein one or more of the plurality of functional areas include a point of interest within the user interface.

### Claim 18 (depends on 17)

The system of claim 17, wherein the sequence of pointer movements are consecutive pointer movements within the user interface of the mobile device, and wherein the map of the sequence of pointer movements corresponds to a pointer trajectory.

### Claim 19 (depends on 16)

The system of claim 16, wherein the instructions are further executable by the at least one processor to cause the system to:
generate, based on the sequence of pointer movements, a set of pointer biometric data, wherein output of the first machine learning model for the movement graph is a set of regression classifiers, and wherein generating the representative set of movement data includes:
performing a dot product of the set of regression classifiers and the sequence of pointer movements.

### Claim 20 (depends on 16)

The system of claim 16, wherein the second, different machine learning model is an extreme gradient boosting (XGBoost) classifier.