Distributed llm framework ecosystem

One or more hardware storage devices that store instructions that are executable by one or more processors of an edge device to cause the one or more processors to:

• receive a user query at the edge device, wherein the one or more hardware storage devices of the edge device include a library of one or more context-specific large language models (LLMs), and wherein each context-specific LLM in the library is respectively trained on only a single corresponding context such that one or more different contexts are represented by the one or more context-specific LLMs;
• determine a context of the user query by semantically analyzing language included in the user query;
• based on the context, identify a context-specific LLM within the library, wherein the context-specific LLM is trained to answer queries having contexts that are the same as the context of the user query;
• load the context-specific LLM into memory of the edge device;
• and
• cause the context-specific LLM to generate an answer to the user query.

An edge device comprising:

• one or more processors;
• and
• one or more hardware storage devices that store instructions that are executable by the one or more processors to cause the edge device to:
• receive a user query, wherein the one or more hardware storage devices include a library of one or more context-specific large language models (LLMs), and wherein each context-specific LLM in the library is respectively trained on only a single corresponding context such that one or more different contexts are represented by the one or more context-specific LLMs;
• determine a context of the user query by semantically analyzing language included in the user query;
• based on the context, identify a context-specific LLM within the library, wherein the context-specific LLM is trained to answer queries having contexts that are the same as the context of the user query;
• load the context-specific LLM into memory of the edge device;
• and
• cause the context-specific LLM to generate an answer to the user query.