Systems and methods for collaborative audio agents using large language models and retrieval-augmented generation

A computing system for diagnosing device issues through a chat session utilizing a Large Language Model (LLM) as an interface between a human user and multiple expert audio models, the computing system comprising:

• one or more processors programmed to:
• receive an audio recording from a human user, wherein the audio recording is associated with a potential error or problem with a device;
• execute an LLM to obtain additional information from the user regarding the audio recording;
• identify a realm of sound based upon the audio recording and the additional input from the user;
• select an expert audio agent specialized in the identified realm of sound, wherein the expert audio agent is one of a plurality of expert audio agents configured to analyze specific types of audio characteristics in a respective one of the realms of sound;
• process the audio recording via the selected expert audio agent utilizing retrieval augmented generation (RAG) to yield data related to a likely cause of the error or problem associated with the audio recording;
• and
• execute the LLM to communicate the likely cause of the error or problem output by the selected expert audio agent to the user.

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method comprising:

• receiving an audio recording from a human user, wherein the audio recording is associated with a potential error or problem with a device;
• executing an LLM to obtain additional information from the user regarding the audio recording;
• identifying a realm of sound based upon the audio recording and the additional input from the user;
• selecting an expert audio agent specialized in the identified realm of sound, wherein the expert audio agent is one of a plurality of expert audio agents configured to analyze specific types of audio characteristics in a respective one of the realms of sound;
• processing the audio recording via the selected expert audio agent utilizing retrieval augmented generation (RAG) to yield data related to a likely cause of the error or problem associated with the audio recording;
• and
• executing the LLM to communicate the likely cause of the error or problem output by the selected expert audio agent to the user.