Multi-topic batching for LLM based text classification

A system for classifying a text, comprising:

• at least one processor executing a code for:
• accessing a text for classification into a class,
• wherein the class is defined as a super-category of a plurality of topics,
• wherein each respective topic of the plurality of topics is associated with at least one prompt for determining whether the text corresponds to the respective topic or not;
• in a plurality of iterations:
• obtaining a batch of a plurality of prompts, wherein the plurality of prompts correspond to the plurality of topics of the class, wherein the plurality of topics are obtained from topics defined as being in an active state;
• feeding the batch to a large language model (LLM);
• analyzing replies generated by the LLM in response to the batch to determine whether the text is classified into each respective topic of the plurality of topics or not;
• changing from the active state to an inactive state for a subset of the plurality of topics to which the text is classified into respective topics according to the analysis,
• wherein in a subsequent iteration another batch is selected from topics remaining in the active state;
• and
• classifying the text into the class or not classifying the text into the class according to an analysis of an aggregation of the subset of the plurality of topics for which the active state was changed to the inactive state;
• wherein at least one prompt associated with each respective topic comprises at least one disqualifying prompt designed to disqualify the text from corresponding to the respective topic and at least one qualifying prompt designed to verify that the text corresponds to the respective topic;
• and
• wherein analyzing the replies generated by the LLM comprises determining whether each reply generated by the LLM disqualifies the text for the respective topic or verifies that the text corresponds to the respective topic, and changing the state of the respective topic to the inactive state in response to the determining indicating that the text is disqualified or verified.

A non-transitory medium storing program instructions for classifying a text, which when executed by at least one processor, cause the at least one processor to:

• access a text for classification into a class,
• wherein the class is defined as a super-category of a plurality of topics, wherein each respective topic of the plurality of topics is associated with at least one prompt for determining whether the text corresponds to the respective topic or not;
• in a plurality of iterations:
• obtain a batch of a plurality of prompts, wherein the plurality of prompts correspond to the plurality of topics of the class, wherein the plurality of topics are obtained from topics defined as being in an active state;
• feed the batch to a large language model (LLM);
• analyze replies generated by the LLM in response to the batch to determine whether the text is classified into each respective topic of the plurality of topics or not;
• change from the active state to an inactive state for a subset of the plurality of topics to which the text is classified into respective topics according to the analysis,
• wherein in a subsequent iteration another batch is selected from topics remaining in the active state;
• and
• classify the text into the class or not classifying the text into the class according to an analysis of an aggregation of the subset of the plurality of topics for which the active state was changed to the inactive state;
• wherein at least one prompt associated with each respective topic comprises at least one disqualifying prompt designed to disqualify the text from corresponding to the respective topic and at least one qualifying prompt designed to verify that the text corresponds to the respective topic;
• and
• wherein analyze replies generated by the LLM comprises determining whether each reply generated by the LLM disqualifies the text for the respective topic or verifies that the text corresponds to the respective topic, and changing the state of the respective topic to the inactive state in response to the determining indicating that the text is disqualified or verified.