Generative language model unlearning

A method for use with a computing system, the method comprising:

• receiving a forgetting dataset including a plurality of forgetting-target prompt-output pairs, wherein the forgetting-target prompt-output pairs each include:
• a forgetting-target prompt that has been input into a generative language model;
• and
• a forgetting-target output generated at the generative language model in response to receiving the forgetting target prompt;
• receiving a remembering dataset including a plurality of remembering-target prompt-output pairs, wherein the remembering-target prompt-output pairs each include:
• a remembering-target prompt that has been input into the generative language model;
• and
• a remembering-target output generated at the generative language model in response to receiving the remembering-target prompt;
• computing an unlearning loss term based at least in part on the forgetting dataset;
• computing a remembering loss term based at least in part on the remembering dataset;
• computing a random mismatch loss term based at least in part on the plurality of forgetting-target prompts and a corresponding plurality of random outputs;
• and
• over a plurality of training steps, performing a respective plurality of unlearning updates at the generative language model by performing gradient descent with respect to a loss that includes the unlearning loss term, the remembering loss term, and the random mismatch loss term.

A computing system comprising:

• one or more processing devices configured to:
• receive a forgetting dataset including a plurality of forgetting-target prompt-output pairs, wherein the forgetting-target prompt-output pairs each include:
• a forgetting-target prompt that has been input into a generative language model;
• and
• a forgetting-target output generated at the generative language model in response to receiving the forgetting target prompt;
• receive a remembering dataset including a plurality of remembering-target prompt-output pairs, wherein the remembering-target prompt-output pairs each include:
• a remembering-target prompt that has been input into the generative language model;
• and
• a remembering-target output generated at the generative language model in response to receiving the remembering-target prompt;
• compute an unlearning loss term based at least in part on the forgetting dataset, wherein the unlearning loss term is a gradient ascent term that is less than or equal to zero;
• compute a remembering loss term based at least in part on the remembering dataset;
• compute a random mismatch loss term based at least in part on the plurality of forgetting-target prompts and a corresponding plurality of random outputs selected at random from among the plurality of remembering-target outputs;
• and
• over a plurality of training steps, perform a respective plurality of unlearning updates at the generative language model by performing gradient descent with respect to a loss that includes the unlearning loss term the remembering loss term, and the random mismatch loss term.