Few-shot continual learning with task-specific parameter selection

One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

• executing a first attention unit included in a transformer neural network to convert a first input token into a first query, a first key, and a first plurality of values, wherein each value included in the first plurality of values represents a sub-task associated with the transformer neural network;
• computing a first plurality of outputs associated with the first input token based on the first query, the first key, and the first plurality of values;
• and
• performing a task associated with an input corresponding to the first input token based on the first input token and the first plurality of outputs.

A system, comprising:

• one or more memories that store instructions, and
• one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of:
• executing a first attention unit included in a transformer neural network to convert a first input token into a first query, a first key, and a first plurality of values, wherein each value included in the first plurality of values represents a sub-task associated with the transformer neural network;
• computing a first plurality of outputs associated with the first input token based on the first query, the first key, and the first plurality of values;
• and
• performing a task associated with an input corresponding to the first input token based on the first input token and the first plurality of outputs.