Prompt caching in generative response engines

A computing device for caching prompts for a generative response engine, comprising:

• at least one memory;
• and
• at least one processor coupled to the at least one memory and configured to:
• receive, by the computing device, a request including a first prompt including a natural language task to perform, wherein the request include an access key for accessing a cloud computing service;
• identify a generative response engine for generating a response to the natural language task based on contents of the natural language task;
• transmit the first prompt and a hash to the generative response engine;
• and
• receive the response to the natural language task from the generative response engine, the response including a number of input tokens,
• wherein the generative response engine is configured to identify a second prompt corresponding with the first prompt using the hash, and wherein the generative response engine determines if activated tokens associated with the second prompt can be loaded for inference of the first prompt.