# US 2024/0403087 A1 · Systems and methods for creating autonomous agents for testing interactive software applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0403087 A1
- Document ID: US20240403087A1
- Status: Published application
- Kind: A1
- Application number: 18/204,121
- Filed: 2023-05-31
- Published: 2024-12-05
- Earliest claimed: 2023-05-31
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Christopher Ward DAVIS; Logan Bruce JONES; Andrea TREVIÑO GAVITO; Haiyan ZHANG; Yanjie HE; Carleton Quarmon HALL; Julia KAHNFELD
- Web page: https://patentgrep.com/patent/US20240403087A1

## CPC

- G06F 9/455: Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06N 3/092: Reinforcement learning
- A63F 13/35: Details of game servers
- A63F 13/60: Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
- A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
- G06F 11/3688: {for test execution, e.g. scheduling of test suites}
- G06F 11/3696: {Methods or tools to render software testable}
- G06N 3/0475: Generative networks
- G06N 5/048: Fuzzy inferencing
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

A system may obtain an emulator. A system may obtain a state detector. A system may receive one or more objective inputs at a large language model. A system may create a decision engine with the large language model and the objective inputs. A system may obtain at least one of video information, audio information, and software state data from the interactive software application at the state detector. A system may transmit state information from the state detector to the decision engine based at least partially on the at least one of video information, audio information, and software state data. A system may select an action with the decision engine in response to the state information. A system may transmit the action to the emulator. A system may transmit at least one emulated input of the action to the interactive software application.

## Claims

### Claim 1

A method of creating an autonomous agent for interacting with an interactive software application, the method comprising:
obtaining an emulator;
obtaining a state detector;
receiving one or more objective inputs at a large language model;
creating a decision engine with the large language model and the objective inputs;
obtaining at least one of video information, audio information, and software state data from the interactive software application at the state detector;
transmitting state information from the state detector to the decision engine based at least partially on the at least one of video information, audio information, and software state data;
selecting an action with the decision engine in response to the state information;
transmitting the action to the emulator; and
transmitting at least one emulated input of the action to the interactive software application.

### Claim 2 (depends on 1)

The method of claim 1, further comprising obtaining an application module including one or more of visual cues, audio cues, objects, textures, characters, animations, and events of the interactive software applications.

### Claim 3 (depends on 2)

The method of claim 2, wherein the application module includes at least one action of the interactive software application.

### Claim 4 (depends on 1)

The method of claim 1, wherein the objective input is a linguistic objective input.

### Claim 5 (depends on 1)

The method of claim 1, wherein the emulator includes an imitation learning model.

### Claim 6 (depends on 1)

The method of claim 1, wherein the state detector includes a machine vision model.

### Claim 7 (depends on 6)

The method of claim 6, wherein the machine vision model compares a first frame of the video information to a second frame of the video information.

### Claim 8 (depends on 1)

The method of claim 1, wherein creating a decision engine with the large language model and the objective inputs includes the large language model creating a fuzzy cognitive map.

### Claim 9 (depends on 1)

The method of claim 1, wherein creating a decision engine with the large language model and the objective inputs includes the large language model creating a fuzzy cognitive map library including a plurality of fuzzy cognitive maps.

### Claim 10

A system for interacting with an interactive software application, the system comprising:
an agent computing device including:
a decision engine configured to select an action in response to state information;
a state detector that provides state information to the decision engine, wherein the state information is based at least partially on video information, audio information, and software state data from the interactive software application; and
an emulator in communication with the decision engine, wherein the emulator generates emulated inputs in response to the action selected by the decision engine.

### Claim 11 (depends on 10)

The system of claim 10, further comprising a large language model, wherein the large language model is configured to receive linguistic objective inputs and provide objectives to the decision engine.

### Claim 12 (depends on 11)

The system of claim 11, wherein the agent computing device further includes the large language model.

### Claim 13 (depends on 10)

The system of claim 10, wherein the decision engine includes a fuzzy cognitive map with input nodes that receive the state information and activation nodes correlated to actions.

### Claim 14 (depends on 13)

The system of claim 13, wherein the decision engine includes a fuzzy cognitive map library including a plurality of fuzzy cognitive maps.

### Claim 15 (depends on 10)

The system of claim 10, wherein the emulator includes an imitation learning model.

### Claim 16 (depends on 10)

The system of claim 10, further comprising an application module including one or more of visual cues, audio cues, objects, textures, characters, animations, and events of the interactive software applications and at least one action of the interactive software application.

### Claim 17

A method of interacting with an interactive software application instantiating an autonomous agent;
obtaining at least one of video information, audio information, and software state data from the interactive software application at a state detector of the autonomous agent;
transmitting state information from the state detector to a decision engine of the autonomous agent based at least partially on the at least one of video information, audio information, and software state data;
selecting an action with the decision engine in response to the state information;
transmitting the action to an emulator of the autonomous agent; and
transmitting at least one emulated input of the action to the interactive software application.

### Claim 18 (depends on 17)

The method of claim 17, wherein transmitting at least one emulated input includes a reaction delay before the at least one emulated input.

### Claim 19 (depends on 17)

The method of claim 17, transmitting at least one emulated input includes transmitting a series of emulated inputs or simultaneous emulated inputs.

### Claim 20 (depends on 19)

The method of claim 19, wherein transmitting a series of emulated inputs includes an intra-input delay in the series of emulated inputs.
