Generating test functions for quantitative prompt evaluation and refinement

Abstract

Certain aspects of the disclosure provide a method for generating test functions for prompt evaluation and refinement. In aspects, the method includes receiving a target prompt from a user; generating a first prompt for causing a language model to generate a task description associated with the target prompt; providing the first prompt to the language model; receiving the task description associated with the target prompt from the language model; generating a second prompt for causing the language model to generate, based on the task description, a set of test functions for quantitatively assessing outputs of the target prompt; providing the second prompt to the language model; receiving the set of test functions from the language model; and returning the set of test functions to the user.

Dependencies are inferred from the claim text.