Combining data selection and reward functions for tuning large language models using reinforcement learning

A computer program product for tuning large language models, the computer program product comprising a computer readable storage medium having program instructions stored therewith, the program instructions executable by one or more processors, the program instructions executable to:

• receive pairs of textual prompts and ground truth labels;
• create a data selection scoring function, by repurposing one or more reward functions to compute similarity between the textual prompts and the ground truth labels, wherein the one or more reward functions measure similarity between textual outputs produced by a large language model and the ground truth labels;
• select a training dataset from the pairs of the textual prompts and the ground truth labels, by using the data selection scoring function;
• and
• tune the large language model using the training dataset and reinforcement learning with the one or more reward functions.

A computer system for tuning large language models, the computer system comprising one or more processors, one or more computer readable tangible storage devices, and program instructions stored on at least one of the one or more computer readable tangible storage devices for execution by at least one of the one or more processors, the program instructions executable to:

• receive pairs of textual prompts and ground truth labels;
• create a data selection scoring function, by repurposing one or more reward functions to compute similarity between the textual prompts and the ground truth labels, wherein the one or more reward functions measure similarity between textual outputs produced by a large language model and the ground truth labels;
• select a training dataset from the pairs of the textual prompts and the ground truth labels, by using the data selection scoring function;
• and
• tune the large language model using the training dataset and reinforcement learning with the one or more reward functions.

A computer-implemented method for tuning large language models, the computer-implemented method comprising:

• receiving pairs of textual prompts and ground truth labels;
• receiving one or more predefined reward functions measuring similarity between textual outputs produced by a large language model and the ground truth labels, in response to determining that the one or more pre-defined reward functions are available;
• creating one or more reward functions measuring the similarity between the textual outputs and the ground truth labels, in response to determining that the one or more pre-defined reward functions are not available;
• creating a data selection scoring function, by repurposing the one or more predefined reward functions or the one or more reward functions to compute similarity between the textual prompts and the ground truth labels;
• selecting a training dataset from the pairs of the textual prompts and the ground truth labels, by using the data selection scoring function;
• and
• tuning the large language model using the training dataset and reinforcement learning with the one or more predefined reward functions or the one or more reward functions.

A computer program product for tuning large language models, the computer program product comprising a computer readable storage medium having program instructions stored therewith, the program instructions executable by one or more processors, the program instructions executable to:

• receive pairs of textual prompts and ground truth labels;
• receive one or more predefined reward functions measuring similarity between textual outputs produced by a large language model and the ground truth labels, in response to determining that the one or more pre-defined reward functions are available;
• create one or more reward functions measuring the similarity between the textual outputs and the ground truth labels, in response to determining that the one or more pre-defined reward functions are not available;
• create a data selection scoring function, by repurposing the one or more predefined reward functions or the one or more reward functions to compute similarity between the textual prompts and the ground truth labels;
• select a training dataset from the pairs of the textual prompts and the ground truth labels, by using the data selection scoring function;
• and
• tune the large language model using the training dataset and reinforcement learning with the one or more predefined reward functions or the one or more reward functions.