Classifying failure modes of large language models (llms) for computer network analytics

An apparatus, comprising:

• one or more network interfaces;
• a processor coupled to the one or more network interfaces and configured to execute one or more processes;
• and
• a memory configured to store a process that is executable by the processor, the process when executed configured to:
• use a large language model associated with a network controller for a computer network to generate answers to questions regarding the computer network;
• determine that a particular answer to one of the questions represents a failure of the large language model;
• classify the failure of the large language model as belonging to particular type of failure;
• and
• provide an indication of the particular type of failure for display.

A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:

• using, by the device, a large language model associated with a network controller for a computer network to generate answers to questions regarding the computer network;
• determining, by the device, that a particular answer to one of the questions represents a failure of the large language model;
• classifying, by the device, the failure of the large language model as belonging to particular type of failure;
• and
• providing, by the device, an indication of the particular type of failure for display.