Multi-tiered cache system

A method comprising:

• in response to receiving an utterance from a user device, providing, by a computer system, the utterance to a multi-tiered cache system associated with an artificial intelligence (AI) model, wherein the multi-tiered cache system comprises a plurality of cache modules;
• determining, by the computer system, a first cache miss for a first cache module in the plurality of cache modules based on the utterance;
• generating, by the computer system, embeddings based on the utterance;
• in response to determining the cache miss for the first cache module, providing, by the computer system, the utterance to a second cache module in the plurality of cache modules;
• obtaining, by the computer system, a response from one of the second cache module or the AI model;
• and
• updating, by the computer system, the first cache module based on the response.

A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:

• receiving an utterance from a user device;
• providing the utterance to a first cache module of a cache system associated with an artificial intelligence (AI) model;
• determining that a first cache hit does not occur at the first cache module based on the utterance;
• subsequent to the determining that the first cache hit does not occur at the first cache module, providing the utterance to a second cache module of the cache system;
• determining whether a second cache hit occurs at a second cache module based on the utterance;
• obtaining a response from one of the second cache module or the AI model;
• and
• providing the response to the user device.