Evaluating the performance of a large language model

A system, comprising:

• a processor configured to:
• receive a query;
• evaluate a performance of a large language model that generated a query response based on the received query, wherein evaluating the performance of the large language model includes determining a reference dependent evaluator score and/or a reference free evaluator score, wherein the reference free evaluator score is based on a coherence value, a relevance value, a consistency value, and a fluency value, wherein the reference free evaluator score is determined in part by providing a second large language model a prompt that includes a chain of thoughts that will instruct the second large language model on how to grade the query response, wherein the prompt indicates a corresponding range of scores for the coherence value, the relevance value, the consistency value, and the fluency value;
• request one or more documents used to generate the received query to be updated based on the reference dependent evaluator score associated with the query response;
• receive the one or more updated documents;
• and
• store the one or more updated documents, wherein the one or more updated documents are utilized by the large language model to generate one or more subsequent query responses for one or more subsequent queries;
• and
• a memory coupled to the processor and configured to provide the processor with instructions.

A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:

• receiving a query;
• evaluating a performance of a large language model that generated a query response based on the received query, wherein evaluating the performance of the large language model includes determining a reference dependent evaluator score and/or a reference free evaluator score, wherein the reference free evaluator score is based on a coherence value, a relevance value, a consistency value, and a fluency value, wherein the reference free evaluator score is determined in part by providing a second large language model a prompt that includes a chain of thoughts that will instruct the second large language model on how to grade the query response, wherein the prompt indicates a corresponding range of scores for the coherence value, the relevance value, the consistency value, and the fluency value;
• requesting one or more documents used to generate the received query to be updated based on the reference dependent evaluator score associated with the query response;
• receiving the one or more updated documents;
• and
• storing the one or more updated documents, wherein the one or more updated documents are utilized by the large language model to generate one or more subsequent query responses for one or more subsequent queries.