Adaptive Query Response System Using Dynamic Language Model Evaluation, Selection, and/or Alignment

One or more non-transitory computer readable media comprising instructions which, when executed by one or more hardware processors, cause performance of operations comprising:

• accessing, by a generative model adaption engine of the adaptive query response system, a conversation, record stored in a data repository of the adaptive query response system, wherein the conversation comprises a present input, one or more initial inputs from a source to an agent and one or more initial responses from the agent to the source, wherein the one or more initial responses are based on one or more initial outputs generated by a first LLM;
• responsive to the conversation record satisfying a criterion by a query evaluator of the adaptive query response system:
• generating, by a prompt generator of the generative model adaption engine, a prompt for a second LLM based on the present input;
• accessing a first output of the second LLM that is output by the second LLM in response to the prompt;
• and
• presenting, in the conversation, an answer based on the first output;
• collecting, by a metric evaluator of the adaptive query response system, a metric based on a property of a subsequent input from the source to the agent in the conversation record;
• evaluating the second LLM based on the metric by updating model performance data associated with the second LLM in the data repository;
• and
• training or fine-tuning, by the adaptive query response system, at least one of the first LLM or the second LLM using feedback derived from the metric and the model performance data, wherein the training adjusts one or more parameters of the at least one LLM to improve subsequent outputs.

A system comprising:

• at least one device including a hardware processor;
• the system being configured to perform operations comprising:
• accessing, by a generative model adaption engine of the adaptive query response system, a conversation, record stored in a data repository of the adaptive query response system, wherein the conversation comprises a present input, one or more initial inputs from a source to an agent and one or more initial responses from the agent to the source, wherein the one or more initial responses are based on one or more initial outputs generated by a first LLM;
• responsive to the conversation record satisfying a criterion by a query evaluator of the adaptive query response system:
• generating, by a prompt generator of the generative model adaption engine, a prompt for a second LLM based on the present input;
• accessing a first output of the second LLM that is output by the second LLM in response to the prompt;
• and
• presenting, in the conversation, an answer based on the first output;
• collecting, by a metric evaluator of the adaptive query response system, a metric based on a property of a subsequent input from the source to the agent in the conversation record;
• evaluating the second LLM based on the metric by updating model performance data associated with the second LLM in the data repository;
• and
• training or fine-tuning, by the adaptive query response system, at least one of the first LLM or the second LLM using feedback derived from the metric and the model performance data, wherein the training adjusts one or more parameters of the at least one LLM to improve subsequent outputs.