Computing systems and methods for llm-based query expansion for use in information retrieval

A system for performing query expansion, the system comprising:

• a memory, a communication interface, and a processor operatively coupled to the memory and the communication interface;
• the processor configured to:
• for each document of a set of documents, use a large language model (LLM) to generate one or more synthetic queries related to the document;
• select one or more synthetic queries related to a query;
• dynamically generate an adaptive few-shot prompt to instruct the LLM to generate a response to the query, wherein the adaptive few-shot prompt comprises an example query-response pair for each of the selected one more synthetic queries;
• provide the adaptive few-shot prompt to the LLM;
• and
• generate an amended query by combining the query and an output of the LLM in response to the adaptive few-shot prompt.

A method for performing query expansion, the method executed in a computing environment comprising one or more processors, a communication interface, and memory, and the method comprising:

• for each document of a set of documents, causing a large language model (LLM) to generate one or more synthetic queries related to the document;
• selecting one or more synthetic queries related to a query;
• dynamically generating an adaptive few-shot prompt to instruct the LLM to generate a response to the query, wherein the adaptive few-shot prompt comprises an example query-response pair for each of the selected one more synthetic queries;
• providing the adaptive few-shot prompt to the LLM;
• and
• generating an amended query by combining the query and an output of the LLM in response to the adaptive few-shot prompt.

A non-transitory computer readable medium storing computer executable instructions which, when executed by at least one computer processor, cause the at least one computer processor to carry out a method for performing query expansion, the method comprising:

• for each document of a set of documents, instructing a large language model (LLM) to generate one or more synthetic queries related to the document;
• selecting one or more synthetic queries related to a query;
• dynamically generating an adaptive few-shot prompt to instruct the LLM to generate a response to the query, wherein the adaptive few-shot prompt comprises an example query-response pair for each of the selected one more synthetic queries;
• providing the adaptive few-shot prompt to the LLM;
• and
• generating an amended query by combining the query and an output of the LLM in response to the adaptive few-shot prompt.