Adaptive information retrieval utilizing semantic and lexical scoring

A method of adaptive information retrieval, comprising:

• receiving, a relevancy search request for document retrieval to augment a prompt to a large language model (LLM);
• identifying a plurality of documents based on a context of the relevancy search request, comprising:
• assigning a semantic score to each respective document of the plurality of documents based on a semantic relevance between the relevancy search request and the respective document;
• and
• assigning a lexical score to each respective document of the plurality of documents based on a lexical relevance between the relevancy search request and the respective document;
• assigning an integrated score for each respective document of the plurality of documents based on the semantic score for the respective document and the lexical score for the respective document, comprising:
• adjusting a weighting of the integrated score of the respective document using an evaluation machine learning model, wherein the weighting imparts increased contextual relevance or lexical accuracy of the respective document to the relevancy search request;
• and
• adjusting a weighting of the semantic score of the respective document based on an index type associated with the relevancy search request;
• ranking each document of the plurality of documents based on the integrated score;
• and
• providing one or more of the plurality of documents to the LLM with the prompt based on a respective ranking of each of the one or more plurality of documents.

An adaptive information retrieval system, comprising:

• a memory comprising computer-executable instructions;
• and a processor configured to execute the computer-executable instructions and cause the adaptive information retrieval system to:
• receive, at the adaptive information retrieval system, a query request for document retrieval;
• identify a plurality of documents based on a context of the query request, wherein to identify the plurality of documents comprises to:
• assign a semantic score to each respective document of the plurality of documents based on a semantic relevance between the query request and the respective document;
• and
• assign a lexical score to each respective document of the plurality of documents based on a lexical relevance between the query request and the respective document;
• assign an integrated score for each respective document of the plurality of documents based on the semantic score for the respective document and the lexical score for the respective document, comprising:
• adjusting a weighting of the integrated score of the respective document using an evaluation machine learning model, wherein the weighting imparts increased contextual relevance or lexical accuracy of the respective document to the query request;
• and
• adjusting a weighting of the semantic score of the respective document based on an index type associated with the query request;
• and
• rank each document of the plurality of documents based on the integrated score.