# US 2025/0158895 A1 · Teaching llm-based agents to troubleshoot networks using reinforcement learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0158895 A1
- Document ID: US20250158895A1
- Status: Published application
- Kind: A1
- Application number: 18/388,242
- Filed: 2023-11-09
- Published: 2025-05-15
- Earliest claimed: 2023-11-09
- Assignees: Cisco Technology, Inc.
- Applicants: Cisco Technology, Inc.
- Inventors: Grégory Mermoud; Pierre-André SAVALLE; Grégoire MAGENDIE; Jean-Philippe VASSEUR; Eduard SCHORNIG
- Web page: https://patentgrep.com/patent/US20250158895A1

## CPC

- H04L 41/16: using machine learning or artificial intelligence
- H04L 41/22: comprising specially adapted graphical user interfaces [GUI]
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

In one implementation, a device uses a large language model-based agent to complete a task with respect to a computer network. The device makes an evaluation of an action taken by the large language model-based agent to complete the task. The device selects, based in part on the evaluation, a set of allowed actions that the large language model-based agent is allowed to perform to complete the task. The device updates the large language model-based agent with an indication of the set of allowed actions that it is allowed to perform to complete the task.

## Claims

### Claim 1

A method comprising:
using, by a device, a large language model-based agent to complete a task with respect to a computer network;
making, by the device, an evaluation of an action taken by the large language model-based agent to complete the task;
selecting, by the device and based in part on the evaluation, a set of allowed actions that the large language model-based agent is allowed to perform to complete the task; and
updating, by the device, the large language model-based agent with an indication of the set of allowed actions that it is allowed to perform to complete the task.

### Claim 2 (depends on 1)

The method as in claim 1, wherein the task comprises troubleshooting an issue in the computer network.

### Claim 3 (depends on 1)

The method as in claim 1, wherein the large language model-based agent completes the task by interfacing with an application programming interface (API) of a controller for the computer network.

### Claim 4 (depends on 1)

The method as in claim 1, wherein the device selects the set of allowed actions based in part on user feedback regarding whether an output of the large language model-based agent from the task was satisfactory.

### Claim 5 (depends on 1)

The method as in claim 1, wherein the large language model-based agent uses a transformer-based policy network to select the action taken by the large language model-based agent to complete the task.

### Claim 6 (depends on 5)

The method as in claim 5, wherein the device updates the transformer-based policy network with the set of allowed actions.

### Claim 7 (depends on 1)

The method as in claim 1, wherein the set of allowed actions comprises an action specified by a subject matter expert via a user interface.

### Claim 8 (depends on 1)

The method as in claim 1, wherein the device makes the evaluation of the action by asking a copy of the large language model-based agent to perform the task in a test network.

### Claim 9 (depends on 1)

The method as in claim 1, wherein the large language model-based agent uses a first large language model to answer a question regarding the task and a second large language model to perform the action.

### Claim 10 (depends on 9)

The method as in claim 9, wherein the second large language model smaller in size than that of the first large language model.

### Claim 11

An apparatus, comprising:
one or more network interfaces;
a processor coupled to the one or more network interfaces and configured to execute one or more processes; and
a memory configured to store a process that is executable by the processor, the process when executed configured to:
use a large language model-based agent to complete a task with respect to a computer network;
make an evaluation of an action taken by the large language model-based agent to complete the task;
select, based in part on the evaluation, a set of allowed actions that the large language model-based agent is allowed to perform to complete the task; and
update the large language model-based agent with an indication of the set of allowed actions that it is allowed to perform to complete the task.

### Claim 12 (depends on 11)

The apparatus as in claim 11, wherein the task comprises troubleshooting an issue in the computer network.

### Claim 13 (depends on 11)

The apparatus as in claim 11, wherein the large language model-based agent completes the task by interfacing with an application programming interface (API) of a controller for the computer network.

### Claim 14 (depends on 11)

The apparatus as in claim 11, wherein the apparatus selects the set of allowed actions based in part on user feedback regarding whether an output of the large language model-based agent from the task was satisfactory.

### Claim 15 (depends on 11)

The apparatus as in claim 11, wherein the large language model-based agent uses a transformer-based policy network to select the action taken by the large language model-based agent to complete the task.

### Claim 16 (depends on 15)

The apparatus as in claim 15, wherein the apparatus updates the transformer-based policy network with the set of allowed actions.

### Claim 17 (depends on 11)

The apparatus as in claim 11, wherein the set of allowed actions comprises an action specified by a subject matter expert via a user interface.

### Claim 18 (depends on 11)

The apparatus as in claim 11, wherein the apparatus makes the evaluation of the action by asking a copy of the large language model-based agent to perform the task in a test network.

### Claim 19 (depends on 11)

The apparatus as in claim 11, wherein the large language model-based agent uses a first large language model to answer a question regarding the task and a second large language model to perform the action.

### Claim 20

A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:
using, by the device, a large language model-based agent to complete a task with respect to a computer network;
making, by the device, an evaluation of an action taken by the large language model-based agent to complete the task;
selecting, by the device and based in part on the evaluation, a set of allowed actions that the large language model-based agent is allowed to perform to complete the task; and
updating, by the device, the large language model-based agent with an indication of the set of allowed actions that it is allowed to perform to complete the task.
