US 12,743,358 B2 · Hierarchical cascade architecture of language models for multi-stage query classification and agent routing

Abstract

The systems and methods disclosed herein orchestrate task execution among autonomous (or semi-autonomous) AI agentic models (“agents”) responsive to a received query by using a hierarchical model cascade to classify queries into agent domains. Queries are processed iteratively by a series of hierarchical levels containing one or more AI models, where each layer is more complex and imposes fewer resource constraints. Each level generates a classification and a confidence score pertaining to the classification. A dynamic bypass mechanism analyzes the classifications and confidence scores at each level to dynamically determine if one or more levels of the hierarchy can be bypassed while resulting in an accurate classification. The final classifications are matched to one or more agents that process the query. Responses from the candidate agents are aggregated into an output that is responsive to the input.

A non-transitory computer-readable storage medium comprising instructions that when executed by at least one data processor of a system, cause the system to:

receive, from a computing device, an output generation request that comprises a query for generation of an output using one or more AI agents of a plurality of AI agents;
transmit a vector representation of the query to a first language model set that is configured to generate (a) a first classification of the query that maps the query to a first subset of the plurality of AI agents and (b) an associated first confidence score;
compare the first confidence score to a first threshold value stored in a threshold data structure accessed by the computing device, the first threshold value determined using a first historical performance metric value set of the first language model set,
wherein failure of the first confidence score to satisfy the first threshold value causes escalation to a second language model set comprising a larger number of model parameters than the first language model set, thereby increasing computational resource usage,
wherein the first threshold value is dynamically updated based on the first historical performance metric value set and a current system load;
in response to a determination that the first confidence score fails to satisfy the first threshold value, transmit the vector representation of the query to the second language model set,
wherein the second language model set is configured to generate (a) a second classification of the query that maps the query to a second subset of the plurality of AI agents and (b) an associated second confidence score;
compare the second confidence score to a second threshold value stored in the threshold data structure accessed by the computing device, the second threshold value determined using a second historical performance metric value set of the second language model set;
in response to a determination that the second confidence score satisfies the second threshold value, aggregate the first classification and the second classification by:
assigning a weight to each of the first classification and the second classification using the first historical performance metric value set and the second historical performance metric value set, respectively, and
generating a composite classification that ranks the first classification and the second classification in accordance with respective weights of the first classification and the second classification;
select a routing AI agent set from the first and second subsets of the plurality of AI agents, wherein each AI agent in the routing AI agent set is configured to autonomously execute a computer-executable task set using the vector representation of the query,
wherein selecting the routing AI agent set comprises:
maintaining a routing control data structure comprising identifiers of AI agents, capabilities of AI agents, performance metric values of the AI agents, and model compliance identifiers of AI agents,
using the composite classification to include, via the routing control data structure, AI agents from the first and second subsets into the routing AI agent set, and
excluding, from the routing AI agent set, one or more AI agents that do not comply with one or more compliance requirements associated with the output generation request, thus preventing transmission of the vector representation of the query to an AI agent whose model compliance identifier fails to satisfy a compliance constraint associated with the output generation request;
and
transmit the vector representation of the query to the routing AI agent set to autonomously execute respective computer-executable task sets.

15 more elements

Hierarchical cascade architecture of language models for multi-stage query classification and agent routing

Life of this patent

Hierarchical cascade architecture of language models for multi-stage query classification and agent routing

Life of this patent

Abstract

Claims20