# US 2022/0036253 A1 · Evaluation of a registration process

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036253 A1
- Document ID: US20220036253A1
- Status: Published application
- Kind: A1
- Application number: 16/941,360
- Filed: 2020-07-28
- Published: 2022-02-03
- Earliest claimed: 2020-07-28
- Assignees: NCS Pearson, Inc.
- Applicants: NCS Pearson, Inc.
- Inventors: Joseph BRUTSCHE; Darrick JENSEN; Michael NEALIS; Peter PASCALE; Vladan PULEC
- Web page: https://patentgrep.com/patent/US20220036253A1

## CPC

- G06Q 10/02: Reservations, e.g. for tickets, services or events
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06Q 10/10: Office automation; Time management
- G06Q 50/205: {Education administration or guidance}
- G06V 10/82: using neural networks
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G09B 7/00: Electrically-operated teaching apparatus or devices working with questions and answers (mechanically operated G09B3/00; computing arrangements G06F)
- G06Q 10/0635: Risk analysis of enterprise or organisation activities
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

Systems and methods may involve processing of entity data by machine learning models to produce one or more entity and/aggregate risk scores and/or aggregate anticipated risk scores, which may be compared to one or more thresholds to determine when one or more predefined actions should be taken. The entity data may be collected for various entities related to an exam registration and delivery process, which may include a candidate, an exam, a test center, an exam registration event, a proctor, and an exam delivery event. The exam registration and delivery process may include multiple states—each being associated with a different set of entities. Aggregate risk scores for a given state may be calculated using only entity data for the set of entities associated with that state. The predetermined actions taken may also be dependent on the current state.

## Claims

### Claim 1

A system comprising:
a data store coupled to a computer network and storing:
a plurality of entity data associated with an exam registration event, and comprising:
an exam registration event data;
a candidate data;
an exam data; and
a test center data, including a trend data associated with each of a plurality of testing facilities identified within the test center data;
an exam registration machine learning model;
a plurality of predefined actions;
a threshold which, if exceeded, triggers a recommendation for at least one predefined action in the plurality of predefined actions; and
a server comprising a server hardware computing device coupled to a network and including at least one processor executing instructions within a memory that, when executed, cause the system to:
receive a notification that the exam registration event is in a real-time exam registration state;
select, from the data store, the exam registration machine learning model, and at least one entity data, in the plurality of entity data, associated with the exam registration event;
perform a risk assessment of the at least one entity data, comprising:
inputting the at least one entity data as a machine learning parameter into the exam registration machine learning model;
receiving, as output from the exam registration machine learning model, an exam registration situational risk score, indicating an exam registration situational risk during, or in response to, the exam registration event associated with, and based on at least one metric derived from, the at least one entity data; and
comparing the exam registration situational risk score to the threshold;
responsive to the exam registration situational risk score exceeding the threshold, update a database entry associated in the data store with the at least one entity data to include the at least one predefined action in the plurality of predefined actions.

### Claim 2 (depends on 1)

The system of claim 1, further comprising a distributed computing system including a plurality of data repositories connected to the data store through the computer network, wherein:
the plurality of entity data is distributed among the data store and the plurality of data repositories; and
the plurality of data repositories are operated by:
an administrative entity operating the system;
a testing facility; or
a trusted third party.

### Claim 3 (depends on 1)

The system of claim 1, wherein the machine learning model is trained via a linear regression analysis or a classification training.

### Claim 4 (depends on 3)

The system of claim 3, further comprising a neural network used by the machine learning model, wherein the neural network includes a plurality of nodes configured to perform the linear regression analysis.

### Claim 5 (depends on 4)

The system of claim 4, further comprising a plurality of known-good testing activity data and a plurality of known-bad testing activity data stored in the data store, wherein the neural network performs the linear regression analysis using a plurality of samples from the plurality of known-good activity data.

### Claim 6

A method comprising:
receiving, by a server comprising a server computing device coupled to a computer network and including a processor executing instructions within a memory, a notification that an exam registration event is in a real-time exam registration state;
selecting, by the server from a data store coupled to the computer network, at least entity data associated with the exam registration event, the at least one entity data being selected from a plurality of entity data stored in the data store and comprising:
an exam registration event data;
a candidate data;
an exam data;
a test center data, including a trend data associated with each of a plurality of testing facilities identified within the test center data; and
performing, by the server, a risk assessment of each of the at least one entity data, comprising:
selecting, from the data store, an exam registration machine learning model stored in the data store;
inputting the at least one entity data as a machine learning parameter into the exam registration machine learning model;
receiving, as output from the exam registration machine learning model, an exam registration situational risk score, indicating an exam registration situational risk during, or in response to, the exam registration event associated with, and based on, at least one metric derived from the at least one entity data; and
comparing the exam registration situational risk score to a threshold stored in the data store which, if exceeded, triggers a recommendation for at least one predefined action in a plurality of predefined actions stored in the data store;
responsive to the situational risk score exceeding the threshold, updating, by the server, a database entry, associated in the data store with the at least one entity data, to include the at least one predefined action in the plurality of predefined actions.

### Claim 7 (depends on 6)

The method of claim 6, wherein the exam registration event data is selected from the group consisting of:
a scheduled exam date for an exam associated with the exam registration event at each of the plurality of test centers;
a scheduled exam time for the exam;
an exam registration event time;
a payment information associated with the exam;
a financial fraud indicator determined based on the payment information;
a number of days that the exam registration occurred prior to the scheduled exam date;
a simultaneous exam registration indicator;
a cross-industry indicator defining whether a first industry to which the exam for which a candidate associated with the testing activity is registering is different from a second industry associated with an exam registration history of the candidate;
at least one unusual frequency indicator defining whether the candidate has repeatedly performed one or more exam registration activities at a frequency exceeding a predetermined frequency threshold stored in the data store;
an authorization indicator defining whether the candidate has been authorized to register for the exam by a third-party.

### Claim 8 (depends on 6)

The method of claim 6, wherein the candidate data for a candidate registering for an exam associated with the exam registration event is selected from the group consisting of:
a name of the candidate;
an email address of the candidate;
a country of residence of the candidate;
a third-party demographic data associated with the candidate;
an Internet Protocol (IP) address associated with a user device operated by the candidate, the IP address being determined by a geolocation of the user device;
a bot behavior indicator associated with the IP address;
a historical fraudulent behavior indicator associated with the candidate;
an account takeover risk indicator associated with the candidate, the account takeover risk indicator being based on an account activity of a user profile associated with the candidate;
an age of the account; and
an exam registration history associated with the candidate.

### Claim 9 (depends on 6)

The method of claim 6, wherein the exam data for an exam associated with the exam registration event comprises:
an exam type for the exam;
a stakes rating for the exam;
an item bank size for an item bank associated with the exam;
a refresh rate for the item bank;
an item bank age for the item bank;
a delivery mechanism for the exam; and
at least one web monitoring result, indicating whether an exam content for the exam has been published online.

### Claim 10 (depends on 6)

The method of claim 6, wherein the test center data is selected from the group consisting of:
a name of a test center associated with the exam registration event;
a country associated with the test center, or in which the test center resides;
an IP address associated with the test center;
a geolocation of the test center, based on the IP address;
an exam registration history for the test center;
a facility type of the test center;
an age of the test center;
a software update compliance indicator for the test center;
an exam result pattern indicator for the test center;
a registration pattern indicator for the test center;
a geolocation mismatch indicator for the test center; and
a number of test centers located within a predefined radius from the test center.

### Claim 11 (depends on 10)

The method of claim 10, wherein the trend data comprises:
a plurality of scoring trend data for each of the plurality of testing facilities;
a candidate trend data for each of a plurality of candidates associated with at least one exam registration event for participating in a testing activity at each of the plurality of testing facilities;
a plurality of travel pattern trend data for each of the plurality of candidates.

### Claim 12 (depends on 11)

The method of claim 11, wherein the trend data comprises:
a plurality of frequency and rate trend data defining a frequency and rate of a plurality of test schedules or test reschedules at each of the plurality of testing facilities;
a plurality of geographically relevant trend data identifying each of a plurality of additional testing facilities within a geographical radius of each of the plurality of testing facilities;
a plurality of exam type trend data identifying at least one exam type trend associated with each of the plurality of testing facilities; and
a plurality of bot traffic pattern trend data identifying at least one bot traffic pattern associated with each of the plurality of testing facilities.

### Claim 13 (depends on 6)

The method of claim 6, wherein the plurality of entity data further comprises a geographic location data associated with the exam registration event, and comprising a country data selected from the group consisting of:
a candidate country in which a candidate associated with the exam registration event is located;
a testing event country in which a testing center in the plurality of testing centers and associated with the exam registration event is located; and
a plurality of zones associated with the exam registration event, and further associated with at least one legal restriction for the exam registration event.

### Claim 14

A system comprising a server, comprising a server computing device coupled to a computer network and including at least one processor executing instructions within a memory which, when executed, cause the system to:
receive a notification that an exam registration event is in a real-time exam registration state;
select, from a data store coupled to the computer network, at least one entity data associated with the testing activity, the at least one entity data being selected from a plurality of entity data stored in the data store and comprising:
an exam registration event data;
a candidate data;
an exam data; and
a test center data, including a trend data associated with each of a plurality of testing facilities identified within the test center data;
perform a risk assessment of each of the at least one entity data, comprising:
selecting, from the data store, an exam registration machine learning model stored in the data store;
inputting the at least one entity data as a machine learning parameter into the exam registration machine learning model;
receiving, as output from the exam registration machine learning model, an exam registration situational risk score, indicating an exam registration situational risk during, or in response to, the exam registration event associated with, and based on at least one metric derived from, the at least one entity data; and
comparing the exam registration situational risk score to a threshold stored in the data store which, if exceeded, triggers a recommendation for at least one predefined action in a plurality of predefined actions stored in the data store;
responsive to the exam registration situational risk score exceeding the threshold, updating, by the server, a database entry, associated in the data store with the at least one entity data, to include the at least one predefined action in the plurality of predefined actions.

### Claim 15 (depends on 14)

The system of claim 14, wherein the server is further configured, responsive to the exam registration situational risk score exceeding the threshold, to automatically update the database entry to recommend an investigation to determine a performance score for each of the plurality of testing facilities, the investigation comprising registering an individual to take an exam at a test center associated with the exam registration event to collect information about the test center to determine whether the test center is facilitating cheating or other unfair or fraudulent practices resulting in the test center losing a certification or being given at least one additional limitation on test administration of at least one exam of a particular type of category.

### Claim 16 (depends on 14)

The system of claim 14, wherein the server is further configured, responsive to the exam registration situational risk score exceeding the threshold, to automatically update the database entry to initiate an investigation, by a human investigator, of the exam registration situational risk score and the at least one entity data associated with a candidate, an exam, or a test center associated with the exam registration event.

### Claim 17 (depends on 14)

The system of claim 14, wherein the server is further configured, responsive to the exam registration situational risk score exceeding the threshold, to automatically update the database entry to invalidate a testing facility in the plurality of testing facilities, thereby preventing the testing facility from administering a specific type or class of exam.

### Claim 18 (depends on 14)

The system of claim 14, wherein the server is further configured, responsive to the exam registration situational risk score exceeding the threshold, to automatically update the database entry to require a test center to provide an enhanced identity verification comprising:
a first requirement for a photo identification for a candidate associated with the testing activity; or
a second requirement for at least two forms of identification for the candidate.

### Claim 19 (depends on 18)

The system of claim 18, wherein the enhanced identity verification further comprises:
a two-factor authentication; or
a biometric analysis.

### Claim 20 (depends on 14)

The system of claim 14, wherein the server is further configured, responsive to the exam registration situational risk score exceeding the threshold, to automatically update the database entry to:
block an IP address for a candidate associated with the exam registration event; or
block the candidate from registering for a testing event associated with the exam registration event.