# US 2024/0004737 A1 · Evaluation and adaptive sampling of agent configurations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004737 A1
- Document ID: US20240004737A1
- Status: Published application
- Kind: A1
- Application number: 17/852,757
- Filed: 2022-06-29
- Published: 2024-01-04
- Earliest claimed: 2022-06-29
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Marco ROSSI
- Web page: https://patentgrep.com/patent/US20240004737A1

## CPC

- G06F 11/3684: {for test design, e.g. generating new test cases}
- G06F 9/547: {Remote procedure calls [RPC]; Web services}
- G06F 11/3409: {for performance assessment}
- G06F 11/3688: {for test execution, e.g. scheduling of test suites}
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/02: Neural networks
- G06N 5/025: Extracting rules from data
- G06N 20/00: Machine learning
- G06N 20/20: Ensemble learning
- G06Q 10/04: Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" (market predictions or forecasting for commercial activities G06Q30/0202)
- G06Q 10/0639: Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q 10/40: Business processes related to social networking or social networking services
- G06Q 30/015: Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
- G06Q 30/0201: Market modelling; Market analysis; Collecting market data
- G06Q 30/0244: {Optimization}
- G06Q 30/0251: Targeted advertisements
- G06Q 30/0631: {Recommending goods or services}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

This document relates to evaluation of automated agents. One example includes a system having a processor and a storage medium. The storage medium can store instructions which, when executed by the processor, cause the system to perform two or more data gathering iterations, which can include distributing experimental units to a plurality of agents having different agent configurations according to a sampling strategy, populating an event log with events representing reactions of an environment to actions taken by individual agents in response to individual experimental units, and adjusting the sampling strategy for use in a subsequent data gathering iteration based at least on the events in the event log. The event log can provide a basis for subsequent evaluation of the plurality of agents with respect to one or more evaluation metrics.

## Claims

### Claim 1

A method comprising:
performing two or more data gathering iterations comprising:
distributing experimental units to a plurality of agents having different agent configurations, the experimental units being distributed according to a sampling strategy;
populating an event log with events representing reactions of an environment to actions taken by individual agents in response to individual experimental units; and
based at least on the events in the event log, adjusting the sampling strategy for use in a subsequent data gathering iteration;
based at least on the events in the event log, predicting performance of the plurality of agents with respect to one or more evaluation metrics; and
based at least on predicted performance of the plurality of agents with respect to the one or more evaluation metrics,
identifying a selected agent configuration.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
deploying a selected agent having the selected agent configuration.

### Claim 3 (depends on 2)

The method of claim 2, wherein the selected agent configuration is selected automatically or based on user input identifying the selected agent configuration from a graphical representation of the predicted performance of the plurality of agents.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
determining importance weights of the individual agents based at least on corresponding probabilities that individual agents give to the actions relative to probabilities of the actions taken by other agents that are stored in the event log; and
adjusting the sampling strategy based at least on the importance weights.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
calculating respective sampling probabilities for the individual agents based at least on the importance weights.

### Claim 6 (depends on 4)

The method of claim 4, wherein adjusting the sampling strategy comprises:
removing at least one agent from subsequent data gathering iterations based at least on the importance weights.

### Claim 7 (depends on 1)

The method of claim 1, wherein the sampling strategy is adjusted at each data gathering iteration based at least on the predicted performance of the plurality of agents with respect to the one or more evaluation metrics.

### Claim 8 (depends on 7)

The method of claim 7, wherein adjusting the sampling strategy comprises:
determining respective confidence intervals of the one or more evaluation metrics for each of the plurality of agents; and
calculating sampling probabilities of individual agents based at least upper bounds of the confidence intervals.

### Claim 9 (depends on 7)

The method of claim 7, wherein adjusting the sampling strategy comprises:
removing at least one agent from further sampling based at least on the predicted performance.

### Claim 10 (depends on 7)

The method of claim 7, further comprising:
populating a data structure with predicted aggregate values and corresponding confidence intervals for the one or more evaluation metrics;
outputting a graphical representation of the data structure; and
identifying one or more agent configurations to sample in a subsequent data gathering iteration based at least on user input directed to the graphical representation of the data structure.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
receiving user input specifying two or more evaluation metrics; and
generating the graphical representation based at least on the two or more evaluation metrics specified by the user input.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
using the events in the event log, predicting performance of at least one other agent with respect to the one or more evaluation metrics, wherein the at least one other agent was not sampled when populating the event log.

### Claim 13

A system comprising:
a processor; and
a storage resource storing instructions which, when executed by the processor, cause the system to:
perform two or more data gathering iterations comprising:
distributing experimental units to a plurality of agents having different agent configurations, the experimental units being distributed according to a sampling strategy;
populating an event log with events representing reactions of an environment to actions taken by individual agents in response to individual experimental units; and
based at least on the events in the event log, adjusting the sampling strategy for use in a subsequent data gathering iteration,
wherein the event log provides a basis for subsequent evaluation of the plurality of agents with respect to one or more evaluation metrics.

### Claim 14 (depends on 13)

The system of claim 13, wherein the individual agents include machine learning agents having different hyperparameters or different feature definitions.

### Claim 15 (depends on 13)

The system of claim 13, wherein the individual agents include at least two different reinforcement learning agents having different reward functions, at least two different supervised learning agents having different loss functions, and at least two different rule-based agents having different rules.

### Claim 16 (depends on 13)

The system of claim 13, wherein the sampling strategy is based at least on respective importance weights of the individual agents.

### Claim 17 (depends on 13)

The system of claim 13, wherein the sampling strategy is adjusted based at least on predicted performance of the plurality of agents with respect to the one or more evaluation metrics.

### Claim 18 (depends on 13)

The system of claim 13, wherein adjusting the sampling strategy comprises assigning respective probabilities to individual agents and randomly assigning the experimental units to the individual agents based on the respective probabilities.

### Claim 19

A computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform acts comprising:
obtaining an event log of events representing reactions of an environment to actions taken by a plurality of agents in response to individual experimental units;
predicting performance of individual agents with respect to one or more evaluation metrics based at least on respective events in the event log reflecting respective actions taken by other agents;
based at least on predicted performance of the individual agents with respect to the one or more evaluation metrics, identifying a selected agent configuration; and
deploying a selected agent having the selected agent configuration.

### Claim 20 (depends on 19)

The computer-readable storage medium of claim 19, wherein the events of the event log are previously sampled using an adaptive sampling strategy that adjusts sampling probabilities of respective agents based on collected events.
