Unlearning a training example from a model

A method of unlearning a training example from a neural network, comprising:

• during training of the neural network on a training dataset, recording a plurality of recordings in a recording dataset, wherein a recording includes weights values of the neural network at a time in which the recording is recorded;
• selecting an unlearning training example to unlearn from the neural network;
• providing per recording, a total-loss-change parameter as an overall loss reduction effect of the unlearning training example on at least one different training example,
• computing per recording, a total sum of values or of absolute values, of the total-loss change parameter for the plurality of training examples;
• and
• assigning a weight for each of the plurality of training examples;
• and
• using the weight of each training example to modify its impact on loss computation during further training to account for the removal of the unlearning training example on each of the plurality of training examples' associated loss, said further training computed from a preceding recording and/or from a current recording forward.

A computer implemented method for unlearning a text element from a large language model (LLM), comprising:

• accessing the text element for unlearning,
• wherein the text element is represented as a plurality of tokens;
• generating a plurality of text training examples by changing a token or a series of contiguous tokens by replacing the token or the series of tokens by another token or another series of token by randomly selecting among candidate tokens with a probability above a threshold according to a prediction of the LLM on the text element when the token or the series is masked away from the LLM input;
• and
• training the LLM on the plurality of text training examples to create an adapted LLM for which it is determined that the text element has been sufficiently unlearned.