# US 2026/0093876 A1 · Streamlined Evaluation of Simultaneously Executed Artificial Intelligence Agents

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0093876 A1
- Document ID: US20260093876A1
- Status: Published application
- Kind: A1
- Application number: 19/412,965
- Filed: 2025-12-09
- Published: 2026-04-02
- Earliest claimed: 2024-06-10
- Assignees: Airia LLC
- Applicants: Airia LLC
- Inventors: William DeWesse; Roman Fedoruk; John Manton; Spencer Reagan; Gregory Roberts; Erich Stuntebeck
- Web page: https://patentgrep.com/patent/US20260093876A1

## CPC

- G06F 11/3457: {Performance evaluation by simulation}
- G06F 9/451: Execution arrangements for user interfaces
- G06F 11/3476: {Data logging (G06F11/14, G06F11/2205 take precedence)}
- G06F 11/3672: {Test management}
- G06F 11/3692: {for test results analysis}
- G06F 21/6227: {where protection concerns the structure of data, e.g. records, types, queries}
- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G06N 3/0475: Generative networks
- G06N 20/00: Machine learning
- G06F 8/34: Graphical or visual programming
- G06F 8/35: model driven
- G06F 9/445: Program loading or initiating (bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57)
- G06F 9/468: {Specific access rights for resources, e.g. using capability register}
- G06F 21/121: {Restricting unauthorised execution of programs}
- G06F 21/316: {by observing the pattern of computer usage, e.g. typical user behaviour}
- G06F 21/36: by graphic or iconic representation
- G06F 21/44: Program or device authentication
- G06F 21/45: Structures or tools for the administration of authentication
- G06F 21/50: Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F 21/6209: {to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself}
- G06F 21/6218: {to a system of files or objects, e.g. local or distributed file system or database}
- G06F 2209/5015: Service provider selection
- G06F 2209/5022: Workload threshold
- G06F 2209/503: Resource availability
- G06F 2221/2141: Access rights, e.g. capability lists, access control lists, access tables, access matrices
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06Q 30/0641: {utilising user interfaces specially adapted for shopping}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Systems and methods are described for comparing execution of two or more artificial intelligence (AI) AI agents. A platform can provide a user interface (UI) that allows for selection or creation of multiple AI agents. The AI agents can utilize different agent objects, such as different prompts, datasets, or models. The AI agents can be displayed on a single UI screen, where execution of the AI agents is simultaneously simulated. The same inputs can be provided to the multiple AI agents, and the corresponding outputs can display on screen. The platform can also vectorize and compare the semantic similarity of the outputs, presenting an indication of the semantic similarity on the same UI screen.

## Claims

### Claim at position 1

-20. (canceled)

### Claim 21

A method of comparatively evaluating performance of artificial intelligence (“AI”) agents, comprising:
receiving, at a user interface (“UI”), a selection of a first AI agent;
causing simultaneous display, on a single screen of the UI (“UI screen”), of the first AI agent and a second AI agent, the first AI agent and second AI agent each comprising multiple agent objects, the second AI agent differing from the first AI agent by at least one agent object, wherein the multiple agent objects are simultaneously displayed in an agent topology;
simultaneously executing the first AI agent and the second AI agent with a common series of queries, wherein an agent executor is configured to execute each agent object according to manifest files of the first AI agent and second AI agent, wherein executing each AI agent generates outputs responsive to the series of queries;
collecting, by the agent executor, execution metrics for each executed agent object; and
causing simultaneous display, on the UI screen, of the execution metrics that correspond to execution of each of the multiple agent objects of the first AI agent and the second AI agent, wherein each of the execution metrics is displayed in proximity to the respective agent object from which it was collected.

### Claim 22 (depends on 21)

The method of claim 21, wherein the execution metrics comprise performance metrics and similarity metrics.

### Claim 23 (depends on 22)

The method of claim 22, wherein the performance metrics comprise latency or response time.

### Claim 24 (depends on 22)

The method of claim 22, wherein the performance metrics comprise token usage.

### Claim 25 (depends on 22)

The method of claim 22, wherein the performance metrics comprise memory utilization.

### Claim 26 (depends on 22)

The method of claim 22, wherein the performance metrics comprise graphical processing unit utilization.

### Claim 27 (depends on 22)

The method of claim 22, wherein the performance metrics comprise telemetry data from hosting environments for the multiple agent objects.

### Claim 28 (depends on 22)

The method of claim 22, wherein the performance metrics for a dataset object comprise a number of data chunks retrieved.

### Claim 29 (depends on 22)

The method of claim 22, wherein the similarity metrics comprise a value representing semantic similarity of vectorized outputs between respective agent objects of the first AI agent and the second AI agent.

### Claim 30 (depends on 21)

The method of claim 21, wherein a different hosting region is selected for the first AI agent than for the second AI agent.

### Claim 31 (depends on 21)

The method of claim 21, wherein the second AI agent differs from the first AI agent based on a hosting type, wherein the hosting type comprises one of on premises, hybrid, and cloud.

### Claim 32 (depends on 21)

The method of claim 21, wherein the simultaneous display identifies variations in execution metrics of corresponding agent objects across the first and second AI agents.

### Claim 33 (depends on 21)

The method of claim 21, wherein the at least one agent object comprises a first dataset, a first AI model, or a first prompt package.

### Claim 34 (depends on 21)

The method of claim 21, wherein the execution metrics further comprise a timeout indicator generated during execution of at least one of the agent objects.

### Claim 35 (depends on 21)

The method of claim 21, further comprising displaying a first total of the execution metrics for the first AI agent, and a second total of the execution metrics for the second AI agent.

### Claim 36 (depends on 21)

The method of claim 21, wherein historical metrics are presented on the UI screen for comparison with the execution metrics.

### Claim 37 (depends on 21)

The method of claim 21, wherein the execution metrics further comprise an execution timeline for each agent object, including a start time and an end time, and the UI screen displays the execution timeline in association with the respective agent object.

### Claim 38 (depends on 21)

The method of claim 21, wherein the UI screen displays a visual indicator of a divergence threshold being exceeded by comparing execution metrics of corresponding agent objects of the first AI agent and the second AI agent.

### Claim 39

A system for comparatively evaluating performance of artificial intelligence (“AI”) agents, comprising:
a memory storage including a non-transitory, computer-readable medium comprising instructions; and
at least one hardware-based processor that executes the instructions to carry out stages comprising:
receiving, at a user interface (“UI”), a selection of a first AI agent;
causing simultaneous display, on a single screen of the UI (“UI screen”), of the first AI agent and a second AI agent, the first AI agent and second AI agent each comprising multiple agent objects, the second AI agent differing from the first AI agent by at least one agent object, wherein the multiple agent objects are simultaneously displayed in an agent topology;
simultaneously executing the first AI agent and the second AI agent with a common series of queries, wherein an agent executor is configured to execute each agent object according to manifest files of the first AI agent and second AI agent, wherein executing each AI agent generates outputs responsive to the series of queries;
collecting, by the agent executor, performance metrics for each executed agent object; and
causing simultaneous display, on the UI screen, of the performance metrics that correspond to execution of each of the multiple agent objects of the first AI agent and the second AI agent, wherein each of the performance metrics is displayed in proximity to the respective agent object from which it was collected.

### Claim 40

A non-transitory, computer-readable medium having instructions for comparatively evaluating performance of artificial intelligence (“AI”) agents, the instructions, when executed by a processor, causing the processor to perform stages comprising:
receiving, at a user interface (“UI”), a selection of a first AI agent;
causing simultaneous display, on a single screen of the UI (“UI screen”), of the first AI agent and a second AI agent, the first AI agent and second AI agent each comprising multiple agent objects, the second AI agent differing from the first AI agent by at least one agent object, wherein the multiple agent objects are simultaneously displayed in an agent topology;
simultaneously executing the first AI agent and the second AI agent with a common series of queries, wherein an agent executor is configured to execute each agent object according to manifest files of the first AI agent and second AI agent, wherein executing each AI agent generates outputs responsive to the series of queries;
collecting, by the agent executor, performance metrics for each executed agent object; and
causing simultaneous display, on the UI screen, of the performance metrics that correspond to execution of each of the multiple agent objects of the first AI agent and the second AI agent, wherein each of the performance metrics is displayed in proximity to the respective agent object from which it was collected.
