Input-output token savings using small language models

A system comprising:

• at least one processor;
• and
• at least one non-transitory computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to perform processing comprising:
• receiving a query, wherein processing the query comprises at least a first easy task, a complicated task, and a second easy task;
• processing the first easy task using a small language model (SLM), thereby obtaining a first SLM processing result;
• processing the complicated task using a large language model (LLM), the processing of the complicated task comprising inputting the first SLM processing result to the LLM and processing the first SLM processing result using the LLM, thereby obtaining an LLM processing result that incorporates at least a portion of the first SLM processing result;
• processing the second easy task using the SLM, the processing of the second easy task comprising inputting the LLM processing result to the SLM and processing the LLM processing result using the SLM, thereby obtaining a second SLM processing result that incorporates at least a portion of the LLM processing result;
• generating a query response comprising at least a portion of the the second SLM processing result;
• and
• outputting the query response.

A method comprising:

• receiving, by at least one processor, a query wherein processing the query comprises a plurality of tasks;
• performing, by the at least one processor, a first of the plurality of tasks by providing at least a portion of the query and a first instruction to a small language model (SLM), thereby obtaining a first intermediate processing result;
• performing, by the at least one processor, a second of the plurality of tasks by providing at least a portion of the first intermediate processing result and a second instruction to a large language model (LLM), thereby obtaining a second intermediate processing result;
• performing, by the at least one processor, a third of the plurality of tasks by providing at least a portion of the second intermediate processing result and a third instruction to the SLM, thereby obtaining a third intermediate processing result;
• performing, by the at least one processor, a fourth of the plurality of tasks by providing at least a portion of the third intermediate processing result and a fourth instruction to the LLM, thereby obtaining a fourth intermediate processing result;
• performing, by the at least one processor, a fifth of the plurality of tasks by providing at least a portion of the fourth intermediate processing result and a fifth instruction to the SLM, thereby obtaining a query response;
• and
• outputting, by the at least one processor, the query response.