Dynamic Selection of Machine-Learning Large Language Models Based on Queries

A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform operations comprising:

• obtaining a plurality of queries from users and for each query in the plurality of queries, obtaining a respective model deployment selected for the query among a set of model deployments;
• for each query in the plurality of queries, assigning the query to a respective category among a set of categories by applying one or more machine-learning models to information obtained from the query;
• generating a dataset, wherein for each category in the set of categories, the dataset includes a mapping between the category and a respective model deployment for the category identified based on one or more queries assigned to the category, wherein the dataset is stored in a database;
• receiving, from a client device, a user query;
• assigning the user query to a particular category of the set of categories by applying the one or more machine-learning models to information obtained from the user query;
• identifying a model deployment mapped to the particular category from the database;
• providing the user query to the identified model deployment for execution;
• and
• providing a response obtained from the identified model deployment to the client device as a response to the user query.

A computer system, comprising:

• one or more computer processors;
• and
• a non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform operations comprising:
• obtaining a plurality of queries from users and for each query in the plurality of queries, obtaining a respective model deployment selected for the query among a set of model deployments;
• for each query in the plurality of queries, assigning the query to a respective category among a set of categories by applying one or more machine-learning models to information obtained from the query;
• generating a dataset, wherein for each category in the set of categories, the dataset includes a mapping between the category and a respective model deployment for the category identified based on one or more queries assigned to the category, wherein the dataset is stored in a database;
• receiving, from a client device, a user query;
• assigning the user query to a particular category of the set of categories by applying the one or more machine-learning models to information obtained from the user query;
• identifying a model deployment mapped to the particular category from the database;
• providing the user query to the identified model deployment for execution;
• and
• providing a response obtained from the identified model deployment to the client device as a response to the user query.