Complementary Prompting For Rehearsal-Free Continual Learning

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• obtaining a set of training samples, each training sample in the set of training samples associated with a respective task of a plurality of different tasks;
• obtaining a task-invariant prompt representative of learned knowledge common to each respective task of the plurality of different tasks;
• for each respective task of the plurality of different tasks, obtaining a respective task-specific prompt representative of learned knowledge specific to the respective task;
• and
• during each of one or more training iterations, for each respective training sample in the set of training samples:
• selecting the respective task-specific prompt representative of the respective task of the respective training sample;
• and
• training a model using the task-invariant prompt and the selected respective task-specific prompt.