Compressing Information Provided to a Machine-Trained Model Using Abstract Tokens

A computing system for interacting with a machine-trained model, comprising:

• an instruction data store for storing computer-readable instructions;
• a processing system for executing the computer-readable instructions in the data store, to perform operations including:
• using the machine-trained model to generate a response to a first prompt, the first prompt expressing input information for a current dialogue turn of a dialogue and first abstract token information generated in one or more prior dialogue turns,
• the machine-trained model being trained by successively improving predictive accuracy of the machine-trained model, while rewarding the machine-trained model based on an extent to which the machine-trained model compresses instances of abstract token information,
• each instance of abstract token information expressing a current state of a dialogue history of the dialogue with fewer tokens than the dialogue history.

A computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer-readable instructions to perform operations, the operations comprising:

• using a teacher version of a machine-trained model to a generate a ground-truth response for an input example, the teacher version of the machine-trained model having fixed weights during training;
• using a student version of the machine-trained model to generate a student-model response for the input example;
• using the student version of the machine-trained model to generate student-model abstract token information;
• using the ground-truth response and the student-model response to generate prediction loss;
• using the student-model abstract token information to generate sparsity loss;
• and
• using plural instances of the prediction loss and plural instances of the sparsity loss to update the student version of the machine-trained model.