# US 2023/0054427 A1 · Artificial intelligence driven assessment and feedback tool

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0054427 A1
- Document ID: US20230054427A1
- Status: Published application
- Kind: A1
- Application number: 17/404,224
- Filed: 2021-08-17
- Published: 2023-02-23
- Earliest claimed: 2021-08-17
- Assignees: Myriad Sensors, Inc.
- Applicants: Myriad Sensors, Inc.
- Inventors: CLIFTON ROOZEBOOM; DAVID BAKKER; ROBERT DOUTHITT; JOHN BOWER; CORIN DUBIE
- Web page: https://patentgrep.com/patent/US20230054427A1

## CPC

- G09B 7/02: of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
- G06F 9/453: {Help systems}
- G06F 11/3438: {monitoring of user actions (tracking the activity of the user H04L67/535)}
- G09B 7/06: of the multiple-choice answer-type, i.e. where a given question is provided with a series of answers and a choice has to be made from the answers
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/00: Machine learning
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

In some embodiments, a computer system comprises processors, a request manager coupled to the processors and configured to receive practice test requests, a practice test manager configured to retrieve practice test data, a real time session manager configured to establish a real time session with a user computer, an AI-based analyzer configured to execute a machine learning model to determine accuracy of results received from the user computer, a non-transitory computer-readable storage medium storing sequences of instructions for: receiving, using the request manager, a request for performing a practice test; retrieving, using the practice test manager, data for the practice test; using the real time session manager: establishing a real time session with the user computer to enable the user computer to access the data and execute the practice test; and as the practice test is executed, collecting test results and transmitting them to user devices.

## Claims

### Claim 1

A computer system comprising:
one or more processors;
a request manager coupled to the one or more processors and configured to receive practice test requests;
a practice test manager configured to retrieve and provide practice test data;
a real time session manager configured to establish a real time session with a user computer;
an artificial intelligence (AI) based analyzer configured to execute a machine learning model to determine accuracy of results received from the user computer;
a non-transitory computer-readable storage medium coupled to the one or more processors and storing one or more sequences of instructions which, when executed by the one or more processors, cause the one or more processors to perform:
receiving, using the request manager, a request for performing a practice test;
retrieving, using the practice test manager, data for the practice test;
using the real time session manager:
establishing a real time session with the user computer to enable the user
computer to access the data and execute the practice test; and
as the real time session is in progress and the practice test is executed, collecting test results received from the user computer;
transmitting the test results to one or more user devices.

### Claim 2 (depends on 1)

The computer system of claim 1, storing additional instructions for:
in response to determining that the real time session has ended, analyzing the test results, using the AI-based analyzer executing the machine learning model, to determine one or more quality metrics for the test results;
transmitting the one or more quality metrics to the one or more user devices;
wherein transmitting the one or more quality metrics to the one or more user devices causes a user device, from the one or more user devices, to generate a graphical representation of the one or more quality metrics and display the graphical representation on a display device of the user device.

### Claim 3 (depends on 2)

The computer system of claim 2, wherein displaying the graphical representation on the display device of the user device comprises providing analytic functionalities for analyzing the one or more quality metrics and determining an acumen and comprehension level of a user who performed the practice test and that is determined based on the one or more quality metrics;
wherein displaying the graphical representation on the display device of the user device comprises generating a graphical user interface dashboard that graphically depicts the acumen and comprehension level of the user who performed the practice test;
wherein the graphical user interface dashboard displays one or more of: icons, text boxes, graphs, data arrangements or tables capturing the one or more quality metrics.

### Claim 4 (depends on 1)

The computer system of claim 1, storing additional instruction for:
as the real time session is in progress and the practice test is executed, generating, using the practice test manager, a visual overlay to be displayed over a depiction of data of the practice test;
wherein the visual overlay comprises prompts, graphs, and information that point out trends and relevant data points, and that assist in executing the practice test.

### Claim 5 (depends on 1)

The computer system of claim 1, storing additional instruction for:
as the real time session is in progress and the practice test is executed, collecting sensor signals from one or more sensors communicatively coupled to the user computer, and including the sensor signals in the test results.

### Claim 6 (depends on 3)

The computer system of claim 3, storing additional instruction for:
receiving, using the request manager, a plurality of requests for performing a plurality of practice tests;
retrieving, using the practice test manager, a plurality of data sets for the plurality of practice tests;
using the real time session manager:
establishing a plurality of real time sessions with a plurality of user computers to enable the plurality of user computers to access corresponding data of the plurality of practice tests and execute a corresponding practice test from the plurality of practice tests from the plurality of user computers; and
for each particular real time session, of the plurality of real time sessions:
as the particular real time session is in progress and the corresponding practice test is executed, collecting particular test results received from a particular user computer;
transmitting the particular test results to the one or more user devices.

### Claim 7 (depends on 6)

The computer system of claim 6, storing additional instructions for:
for each particular real time session, of the plurality of real time sessions:
in response to determining that the particular real time session has ended, analyzing the particular test results, using the AI-based analyzer executing the machine learning model, to determine one or more particular quality metrics for the particular test results;
transmitting the one or more particular quality metrics to the one or more user devices;
wherein transmitting the one or more particular quality metrics to the one or more user devices causes a particular user device, from the one or more user devices, to generate a particular graphical representation of the one or more particular quality metrics and display the particular graphical representation on a particular display device of the user device;
wherein displaying the particular graphical representation on the particular display device of the user device comprises providing analytic functionalities for analyzing the one or more particular quality metrics;
wherein displaying the particular graphical representation on the particular display device of the user device comprises generating a particular graphical user interface dashboard that graphically depicts the acumen and comprehension level of the user who performed the corresponding practice test;
wherein the particular graphical user interface dashboard displays one or more of: icons, text boxes, graphs, data arrangements or tables capturing the one or more particular quality metrics.

### Claim 8

A computer-implemented method comprising:
receiving, from a request manager, a request for performing a practice test;
retrieving, using a practice test manager, data for the practice test;
using a real time session manager:
establishing a real time session with a user computer to enable the user computer
to access the data and execute the practice test; and
as the real time session is in progress and the practice test is executed, collecting test results received from the user computer;
transmitting the test results to one or more user devices.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, further comprising:
in response to determining that the real time session has ended, analyzing the test results, using an AI-based analyzer executing a machine learning model, to determine one or more quality metrics for the test results;
transmitting the one or more quality metrics to the one or more user devices;
wherein transmitting the one or more quality metrics to the one or more user devices causes a user device, from the one or more user devices, to generate a graphical representation of the one or more quality metrics and display the graphical representation on a display device of the user device.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein displaying the graphical representation on the display device of the user device comprises providing analytic functionalities for analyzing the one or more quality metrics and determining an acumen and comprehension level of a user who performed the practice test and that is determined based on the one or more quality metrics;
wherein displaying the graphical representation on the display device of the user device comprises generating a graphical user interface dashboard that graphically depicts the acumen and comprehension level of the user who performed the practice test;
wherein the graphical user interface dashboard displays one or more of: icons, text boxes, graphs, data arrangements or tables capturing the one or more quality metrics.

### Claim 11 (depends on 8)

The computer-implemented method of claim 8, further comprising:
as the real time session is in progress and the practice test is executed, generating, using the practice test manager, a visual overlay to be displayed over a depiction of data of the practice test;
wherein the visual overlay comprises prompts, graphs, and information that point out trends and relevant data points, and that assist in executing the practice test.

### Claim 12 (depends on 8)

The computer-implemented method of claim 8, further comprising:
as the real time session is in progress and the practice test is executed, collecting sensor signals from one or more sensors communicatively coupled to the user computer, and including the sensor signals in the test results.

### Claim 13 (depends on 10)

The computer-implemented method of claim 10, further comprising:
receiving, using the request manager, a plurality of requests for performing a plurality of practice tests;
retrieving, using the practice test manager, a plurality of data sets for the plurality of practice tests;
using the real time session manager:
establishing a plurality of real time sessions with a plurality of user computers to enable the plurality of user computers to access corresponding data of the plurality of practice tests and execute a corresponding practice test from the plurality of practice tests from the plurality of user computers; and
for each particular real time session, of the plurality of real time sessions:
as the particular real time session is in progress and the corresponding practice test is executed, collecting particular test results received from a particular user computer;
transmitting the particular test results to the one or more user devices.

### Claim 14 (depends on 13)

The computer-implemented method of claim 13, further comprising:
for each particular real time session, of the plurality of real time sessions:
in response to determining that the particular real time session has ended, analyzing the particular test results, using the AI-based analyzer executing the machine learning model, to determine one or more particular quality metrics for the particular test results;
transmitting the one or more particular quality metrics to the one or more user devices;
wherein transmitting the one or more particular quality metrics to the one or more user devices causes a particular user device, from the one or more user devices, to generate a particular graphical representation of the one or more particular quality metrics and display the particular graphical representation on a particular display device of the user device;
wherein displaying the particular graphical representation on the particular display device of the user device comprises providing analytic functionalities for analyzing the one or more particular quality metrics;
wherein displaying the particular graphical representation on the particular display device of the user device comprises generating a particular graphical user interface dashboard that graphically depicts the acumen and comprehension level of the user who performed the corresponding practice test;
wherein the particular graphical user interface dashboard displays one or more of: icons, text boxes, graphs, data arrangements or tables capturing the one or more particular quality metrics.

### Claim 15

A non-transitory computer-readable storage medium storing one or more computer instructions which, when executed by one or more processors, cause the one or more processors to perform:
receiving, from a request manager, a request for performing a practice test;
retrieving, using a practice test manager, data for the practice test;
using a real time session manager:
establishing a real time session with a user computer to enable the user computer
to access the data and execute the practice test; and
as the real time session is in progress and the practice test is executed, collecting test results received from the user computer;
transmitting the test results to one or more user devices.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, storing additional instructions for:
in response to determining that the real time session has ended, analyzing the test results, using an AI-based analyzer executing a machine learning model, to determine one or more quality metrics for the test results;
transmitting the one or more quality metrics to the one or more user devices;
wherein transmitting the one or more quality metrics to the one or more user devices causes a user device, from the one or more user devices, to generate a graphical representation of the one or more quality metrics and display the graphical representation on a display device of the user device.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein displaying the graphical representation on the display device of the user device comprises providing analytic functionalities for analyzing the one or more quality metrics and determining an acumen and comprehension level of a user who performed the practice test and that is determined based on the one or more quality metrics;
wherein displaying the graphical representation on the display device of the user device comprises generating a graphical user interface dashboard that graphically depicts the acumen and comprehension level of the user who performed the practice test;
wherein the graphical user interface dashboard displays one or more of: icons, text boxes, graphs, data arrangements or tables capturing the one or more quality metrics.

### Claim 18 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, storing additional instructions for:
as the real time session is in progress and the practice test is executed, generating, using the practice test manager, a visual overlay to be displayed over a depiction of data of the practice test;
wherein the visual overlay comprises prompts, graphs, and information that point out trends and relevant data points, and that assist in executing the practice test.

### Claim 19 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, storing additional instructions for:
receiving, using the request manager, a plurality of requests for performing a plurality of practice tests;
retrieving, using the practice test manager, a plurality of data sets for the plurality of practice tests;
using the real time session manager:
establishing a plurality of real time sessions with a plurality of user computers to enable the plurality of user computers to access corresponding data of the plurality of practice tests and execute a corresponding practice test from the plurality of practice tests from the plurality of user computers; and
for each particular real time session, of the plurality of real time sessions:
as the particular real time session is in progress and the corresponding practice test is executed, collecting particular test results received from a particular user computer;
transmitting the particular test results to the one or more user devices.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, storing additional instructions for:
for each particular real time session, of the plurality of real time sessions:
in response to determining that the particular real time session has ended, analyzing the particular test results, using the AI-based analyzer executing the machine learning model, to determine one or more particular quality metrics for the particular test results;
transmitting the one or more particular quality metrics to the one or more user devices;
wherein transmitting the one or more particular quality metrics to the one or more user devices causes a particular user device, from the one or more user devices, to generate a particular graphical representation of the one or more particular quality metrics and display the particular graphical representation on a particular display device of the user device;
wherein displaying the particular graphical representation on the particular display device of the user device comprises providing analytic functionalities for analyzing the one or more particular quality metrics;
wherein displaying the particular graphical representation on the particular display device of the user device comprises generating a particular graphical user interface dashboard that graphically depicts the acumen and comprehension level of the user who performed the corresponding practice test;
wherein the particular graphical user interface dashboard displays one or more of: icons, text boxes, graphs, data arrangements or tables capturing the one or more particular quality metrics.
