Method and System for Multi-Level Artificial Intelligence Supercomputer Design

A method for in-memory processing of large language model (LLM) operations, comprising:

• receiving an input data stream at a data receiver;
• determining an aggregation duration for collecting data from the input data stream;
• aggregating input data from the input data stream over the determined aggregation duration;
• breaking the aggregated input data into data batches sized for in-memory processing;
• processing the data batches in one or more memory modules using multiple LLMs to generate processed output data;
• caching the processed output data in a results cache by an output broker;
• assigning priority scores by the output broker to the cached processed output data based on at least one of:
• frequency of receiving similar input data;
• computational cost of generating the output data;
• and
• recency of the cached processed output data being added to the results cache;
• and
• providing the cached processed output data to subsequently received similar input data based on the assigned priority scores.

A system for processing large language model (LLM) operations, comprising:

• one or more memory modules;
• a non-transitory computer-readable storage medium having software stored thereon that is executable by a processor to:
• operate a data receiver configured to:
• receive an input data stream;
• and
• break the input data stream into batches of input data;
• operate a processing layer comprising:
• one or more in-memory processing components positioned in communication with the one or more memory modules and configured to process the batches of input data in the one or more memory modules;
• a plurality of LLMs configured to process the batches of input data and generate corresponding batches of processed output data;
• and
• an output broker configured to:
• cache results from the plurality of LLMs in a results cache;
• assign priority scores to the cached results;
• rank the cached results based on their priority scores;
• and
• provide ranked results responsive to subsequent similar input data received by the data receiver from the results cache;
• a processor coupled to the one or more memory modules and the storage medium and configured to execute the software stored on the storage medium;
• and
• a communication device operable to send and receive data to and from the processor across a network.