Efficient adaptation of machine learning models

A processor-implemented method for machine learning, comprising:

• receiving an input for processing by a transformer block in a neural network;
• generating an output of the transformer block based on the received input and weights associated with the transformer block;
• generating an output of an adapter associated with the transformer block based on a copy of the received input and adapter weights associated with the adapter;
• storing key-value data associated with the output of the transformer block and key-value data associated with a combination of the output of the transformer block and the output of the adapter in a cache for subsequent inferencing rounds;
• and
• generating a response to the input based on the combination of the output of the transformer block and the output of the adapter.

A processing system for machine learning, comprising:

• means for receiving an input for processing by a transformer block in a neural network;
• means for generating an output of the transformer block based on the received input and weights associated with the transformer block;
• means for generating an output of an adapter associated with the transformer block based on a copy of the received input and adapter weights associated with the adapter;
• means for storing key-value data associated with the output of the transformer block and key-value data associated with a combination of the output of the transformer block and the output of the adapter in a cache for subsequent inferencing rounds;
• and
• means for generating a response to the input based on the combination of the output of the transformer block and the output of the adapter.