# US 2026/0065162 A1 · Proficiency Dashboard System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065162 A1
- Document ID: US20260065162A1
- Status: Published application
- Kind: A1
- Application number: 19/313,822
- Filed: 2025-08-28
- Published: 2026-03-05
- Earliest claimed: 2024-08-28
- Inventors: Ramin Bolouri
- Web page: https://patentgrep.com/patent/US20260065162A1

## CPC

- G06N 20/00: Machine learning
- G06F 11/302: {where the computing system component is a software system}
- G06F 11/32: with visual {or acoustical} indication of the functioning of the machine
- G06F 11/3428: {Benchmarking}
- G06F 11/3438: {monitoring of user actions (tracking the activity of the user H04L67/535)}
- G06F 2201/865: Monitoring of software
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A software system evaluates an artificial intelligence (AI) model across predefined tasks and optional simulated scenarios, computes task-level and aggregated proficiency metrics, stores those metrics keyed to model versions, and displays them on an interactive dashboard featuring real-time updates and side-by-side version comparisons. In certain embodiments, a data capture layer logs user interactions; an incremental training layer updates the model without full retraining; a proficiency scoring module benchmarks performance against human standards; and a versioning module maintains a longitudinal record. The dashboard surfaces strengths, weaknesses, improvements, and regressions and can present fairness/bias indicators and simulation tools for “what-if” testing, thereby increasing transparency and reliability of AI deployments.

## Claims

### Claim 1

A computer-implemented system for tracking and improving proficiency of an artificial intelligence (AI) model, the system comprising: a data capture layer configured to log user interactions and task context during operation of the AI model; an AI training layer communicatively coupled to the data capture layer and configured to incrementally update learned parameters of the AI model using captured context without full retraining; a proficiency scoring module configured to evaluate the AI model on predefined tasks and compute a proficiency score by comparing the AI model's performance to a human benchmark; a versioning module configured to assign version identifiers to successive AI model updates and to record, for each version, the corresponding proficiency score with a timestamp; and a dashboard interface module configured to present a real-time dashboard that displays a current proficiency score and a historical trend across versions and that provides interactive simulation tools enabling a user to specify hypothetical task scenarios and view expected performance of a selected version of the AI model.

### Claim 2 (depends on 1)

The system of claim 1, wherein the data capture layer classifies user actions including corrections, confirmations, and overrides to identify model weaknesses for targeted retraining.

### Claim 3 (depends on 1)

The system of claim 1, wherein the AI training layer performs online or micro-batch updates while preserving prior competencies to reduce catastrophic forgetting.

### Claim 4 (depends on 1)

The system of claim 1, wherein the proficiency scoring module computes category-wise sub-scores that form a composite proficiency score, and the dashboard displays a breakdown across categories.

### Claim 5 (depends on 1)

The system of claim 1, wherein the versioning module triggers an alert upon detecting a proficiency regression beyond a threshold and enables rollback to a prior version.

### Claim 6 (depends on 1)

The system of claim 1, wherein the dashboard updates the displayed proficiency responsive to each completed evaluation run without manual refresh.

### Claim 7 (depends on 1)

The system of claim 1, wherein the simulation tools permit side-by-side comparison of multiple AI model versions on a user-defined scenario.

### Claim 8 (depends on 1)

The system of claim 1, further comprising an explainability panel that identifies captured interactions most influential on recent proficiency changes or provides feature-importance indicators for simulated tasks.

### Claim 9 (depends on 1)

The system of claim 1, wherein the data capture layer anonymizes sensitive information and the repository encrypts logs in transit and at rest, and the dashboard provides authorized audit of data influencing proficiency changes.

### Claim 10

A computer-implemented method comprising: logging user actions and task outcomes during operation of an AI model; selecting training-relevant interactions; incrementally updating the AI model with the selected interactions to form successive versions; evaluating each version on predefined tasks and computing a proficiency score relative to a human benchmark; storing each proficiency score in association with a corresponding version identifier and timestamp; and displaying, via a dashboard, a current proficiency score and a historical trend across versions together with interactive simulation of user-specified scenarios.

### Claim 11 (depends on 10)

The method of claim 10, wherein incremental updates are triggered by thresholds including a volume of new interactions or a detected proficiency drop on recent tasks.

### Claim 12 (depends on 10)

The method of claim 10, further comprising computing fairness metrics by comparing performance across dataset segments and surfacing those metrics on the dashboard.

### Claim 13 (depends on 10)

A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform the method of claim 10.

### Claim 14

A proficiency monitoring system for an AI model, comprising: an evaluation module configured to administer predefined tasks to the AI model and to generate performance data; a data repository configured to store performance data and task-level proficiency metrics keyed to a version identifier; a dashboard interface module configured to display the proficiency metrics; and a version comparison component configured to present a comparative visualization of proficiency metrics for at least two versions of the AI model.

### Claim 15 (depends on 14)

The system of claim 14, further comprising a simulation environment module configured to provide simulated test scenarios whose performance results are evaluated and stored as part of the predefined tasks.

### Claim 16 (depends on 14)

The system of claim 14, wherein the dashboard interface module generates an alert if any proficiency metric falls below a predetermined threshold.

### Claim 17 (depends on 14)

The system of claim 14, wherein the evaluation module computes one or more bias metrics and the dashboard interface displays the bias metrics.

### Claim 18 (depends on 14)

The system of claim 14, wherein tasks are grouped into categories and the proficiency metrics include category aggregates.

### Claim 19 (depends on 14)

The system of claim 14, wherein the repository stores, for each task, input provided to the model, the corresponding model output, and an evaluation result to enable audit.

### Claim 20 (depends on 14)

The system of claim 14, wherein the evaluation module automatically executes tasks and updates the repository upon detecting a newly created or deployed model version.

### Claim 21

A method for monitoring proficiency of an AI model, comprising: providing predefined tasks to the AI model; evaluating model outputs to compute task-level results; computing proficiency metrics from the results; storing the metrics in a repository keyed to a version identifier; and generating a dashboard display that presents the proficiency metrics.

### Claim 22 (depends on 21)

The method of claim 21, further comprising retrieving proficiency metrics of a prior model version, comparing them to those of a current version, and highlighting differences on the dashboard.

### Claim 23 (depends on 21)

The method of claim 21, wherein providing the tasks comprises generating an interactive simulated scenario and evaluating performance within the scenario.

### Claim 24

A non-transitory computer-readable medium storing instructions that, when executed, cause processors to: administer predefined tasks; record performance results; calculate task-level proficiency metrics; store the metrics keyed to a version identifier; and generate a dashboard interface displaying the metrics.
