Efficient model agent communication with cache sharing

A computerized method for context sharing between generative artificial intelligence (GAI) models, the method comprising:

• populating a KV cache of a model with a layer of another KV cache generated by another model;
• recomputing another layer of the KV cache;
• computing an output performance score for the model based on comparison between (a) an output generated by the second model while using the KV cache and (b) a baseline metric;
• and
• transmitting a reuse pattern to a host device for context sharing between an instance of the model and an instance of the other model, the reuse pattern identifying the layer to be reused by the instance of the model, thereby causing the host device to generate a local KV cache for the instance of the model by reusing the layer from the instance of the other model.

A computer storage medium having computer-executable instructions that, upon execution by a processor of a computer, cause the processor to at least:

• identify a pair of generative artificial intelligence (GAI) models including a first model and a second model;
• generate a first key-value (KV) cache of the first model, the first KV cache including a plurality of first KV cache layers;
• populate a second KV cache of the second model with one or more layers of the first KV cache layers;
• recompute one or more other layers of the second KV cache;
• generate an output performance score for the second model based on comparison between (a) output generated by the second model while using the second KV cache and (b) a baseline metric;
• and
• configure a host device with a reuse pattern for context sharing between an instance of the first model and an instance of the second model, the reuse pattern identifying the plurality of first KV cache layers to be reused, thereby causing the host device to generate a local KV cache for the instance of the second model by reusing the plurality of first KV cache layers from a KV cache of the instance of the first model.