# US 2025/0148291 A1 · Using a curriculum for reinforcement learning to train an llm-based network troubleshooting agent > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0148291 A1 - Document ID: US20250148291A1 - Status: Published application - Kind: A1 - Application number: 18/388,010 - Filed: 2023-11-08 - Published: 2025-05-08 - Earliest claimed: 2023-11-08 - Assignees: Cisco Technology, Inc. - Applicants: Cisco Technology, Inc. - Inventors: Grégory Mermoud; Pierre-André SAVALLE; Eduard SCHORNIG; Jean-Philippe VASSEUR - Web page: https://patentgrep.com/patent/US20250148291A1 ## CPC - G06N 3/092: Reinforcement learning - G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] - G06N 20/00: Machine learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract In one implementation, a device may determine how well a large language model-based troubleshooting agent for a network was able to perform during a first test having a first difficulty. The device may update the large language model-based troubleshooting agent using reinforcement learning based on how well the large language model-based troubleshooting agent was able to perform during the first test. The device may select a second difficulty for a second test based on how well the large language model-based troubleshooting agent was able to perform during the first test. The device may initiate the second test to assess how well the large language model-based troubleshooting agent is able to perform. ## Claims ### Claim 1 A method comprising: determining, by a device, how well a large language model-based troubleshooting agent for a network was able to perform during a first test having a first difficulty; updating, by the device, the large language model-based troubleshooting agent using reinforcement learning based on how well the large language model-based troubleshooting agent was able to perform during the first test; selecting, by the device, a second difficulty for a second test based on how well the large language model-based troubleshooting agent was able to perform during the first test; and initiating, by the device, the second test to assess how well the large language model-based troubleshooting agent is able to perform. ### Claim 2 (depends on 1) The method as in claim 1, wherein the first test comprises a particular network scenario and an input request for the large language model-based troubleshooting agent. ### Claim 3 (depends on 2) The method as in claim 2, further comprising: configuring the particular network scenario in the network. ### Claim 4 (depends on 1) The method as in claim 1, wherein the second test has as higher difficulty than that of the first test, based on the large language model-based troubleshooting agent being able to successfully perform the first test. ### Claim 5 (depends on 1) The method as in claim 1, wherein the device selects the second difficulty for the second test further based on a prediction that performance of the large language model-based troubleshooting agent during the second test will lead to selection of a third difficulty for a third test. ### Claim 6 (depends on 1) The method as in claim 1, wherein the first test and the second test comprise a same network scenario instantiated in the network but comprise different input requests for the large language model-based troubleshooting agent. ### Claim 7 (depends on 1) The method as in claim 1, wherein the first test evaluates how well the large language model-based troubleshooting agent was able to troubleshoot a particular impairment scenario in the network. ### Claim 8 (depends on 1) The method as in claim 1, wherein the device generates the first test and the second test using a large language model-based generator. ### Claim 9 (depends on 1) The method as in claim 1, wherein selecting the second difficulty for the second test comprises: using a discriminator to compute a predicted reward value for the first test; and comparing the predicted reward value to an actual reward value that is based on how well the large language model-based troubleshooting agent for a network was able to perform during the first test. ### Claim 10 (depends on 1) The method as in claim 1, wherein the device selects the second difficulty for the second test based in part on a history of previously performed tests. ### Claim 11 An apparatus, comprising: one or more network interfaces; a processor coupled to the one or more network interfaces and configured to execute one or more processes; and a memory configured to store a process that is executable by the processor, the process when executed configured to: determine how well a large language model-based troubleshooting agent for a network was able to perform during a first test having a first difficulty; update the large language model-based troubleshooting agent using reinforcement learning based on how well the large language model-based troubleshooting agent was able to perform during the first test; select a second difficulty for a second test based on how well the large language model-based troubleshooting agent was able to perform during the first test; and initiate the second test to assess how well the large language model-based troubleshooting agent is able to perform. ### Claim 12 (depends on 11) The apparatus as in claim 11, wherein the first test comprises a particular network scenario and an input request for the large language model-based troubleshooting agent. ### Claim 13 (depends on 12) The apparatus as in claim 12, wherein the process when executed is further configured to: configure the particular network scenario in the network. ### Claim 14 (depends on 11) The apparatus as in claim 11, wherein the second test has as higher difficulty than that of the first test, based on the large language model-based troubleshooting agent being able to successfully perform the first test. ### Claim 15 (depends on 11) The apparatus as in claim 11, wherein the apparatus selects the second difficulty for the second test further based on a prediction that performance of the large language model-based troubleshooting agent during the second test will lead to selection of a third difficulty for a third test. ### Claim 16 (depends on 11) The apparatus as in claim 11, wherein the first test and the second test comprise a same network scenario instantiated in the network but comprise different input requests for the large language model-based troubleshooting agent. ### Claim 17 (depends on 11) The apparatus as in claim 11, wherein the first test evaluates how well the large language model-based troubleshooting agent was able to troubleshoot a particular impairment scenario in the network. ### Claim 18 (depends on 11) The apparatus as in claim 11, wherein the apparatus generates the first test and the second test using a large language model-based generator. ### Claim 19 (depends on 11) The apparatus as in claim 11, wherein the apparatus selects the second difficulty for the second test by: using a discriminator to compute a predicted reward value for the first test; and comparing the predicted reward value to an actual reward value that is based on how well the large language model-based troubleshooting agent for a network was able to perform during the first test. ### Claim 20 A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising: determining, by the device, how well a large language model-based troubleshooting agent for a network was able to perform during a first test having a first difficulty; updating, by the device, the large language model-based troubleshooting agent using reinforcement learning based on how well the large language model-based troubleshooting agent was able to perform during the first test; selecting, by the device, a second difficulty for a second test based on how well the large language model-based troubleshooting agent was able to perform during the first test; and initiating, by the device, the second test to assess how well the large language model-13 based troubleshooting agent is able to perform.