Producing a Reduced-Size Model by Explanation Tuning

A computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer-readable instructions to perform a training operation, the training operation comprising:

• submitting a student-model prompt to a student language model, and, in response, receiving a student-model response,
• the student-model prompt expressing a combination of a student-model system instruction and a student-model client instruction,
• the student-model system instruction requesting the student language model to formulate responses to queries that describe student-model final results and processes of producing the student-model final results,
• the student-model client instruction expressing a query,
• the student-model response describing a student-model final result and a process of producing the student-model final result;
• receiving a teacher-model response, the teacher-model response being produced by a teacher language model based on a teacher-model prompt, the teacher-model prompt including a teacher-model system instruction that requests the teacher language model to formulate responses to queries that describe teacher-model final results and processes of producing the teacher-model final results,
• the teacher-model response describing a teacher-model final result and a process of producing the teacher-model final result;
• generating a measure of loss that depends on a difference between the teacher-model response and the student-model response;
• updating parameters of the student language model based on the loss;
• and
• repeating the submitting, receiving, generating, and updating for other prompts.

A system for using a transformer-based client language model, comprising:

• a data store for storing computer-readable instructions;
• a processing system for executing the computer-readable instructions in the data store, to perform operations including:
• receiving a client-model system instruction that requests the transformer-based client language model to formulate responses to queries that describe client-model final results and processes of producing the client-model final results;
• receiving a client-model client instruction that specifies a query;
• producing a client-model prompt that includes a combination of the client-model system instruction and the client-model client instruction;
• and
• submitting the client-model prompt to the transformer-based client language model, the transformer-based client language model transforming the client-model prompt into a client-model response, the client-model response describing a client-model final result and a process of producing the client-model final result via intermediary results,
• the transformer-based client language model producing the client-model response using parameters that are trained based on teacher-model responses produced by a transformer-based teacher language model in response to teacher-model prompts, each teacher-model prompt expressing a combination of a teacher-model system instruction and a teacher-model client instruction, and
• each teacher-model system instruction requesting the transformer-based teacher language model to formulate teacher-model responses to queries that describe teacher-model final results and processes of producing the teacher-model final results.