# US 2025/0094459 A1 · Method and System for Multi-Level Artificial Intelligence Supercomputer Design

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094459 A1
- Document ID: US20250094459A1
- Status: Published application
- Kind: A1
- Application number: 18/965,072
- Filed: 2024-12-02
- Published: 2025-03-20
- Earliest claimed: 2023-05-04
- Assignees: Vijay Madisetti
- Inventors: Vijay Madisetti; Arshdeep Bahga
- Web page: https://patentgrep.com/patent/US20250094459A1

## CPC

- G06F 16/3329: Natural language query formulation
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/30: Semantic analysis
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system for processing LLM operations including a memory module, a data receiver to receive an input data stream and break the input data stream into batches of input data, a processing layer to process the batches of input data in the one or more memory modules and having LLMs to process the batches of input data and generate corresponding batches of processed output data and an output broker to cache results from the plurality of LLMs in a results cache, assign priority scores to the cached results, rank the cached results based on their priority scores, and provide ranked results responsive to subsequent similar input data received by the data receiver from the results cache.

## Claims

### Claim 1

A system for processing large language model (LLM) operations, comprising:
one or more memory modules;
a data receiver configured to:
receive an input data stream; and
break the input data stream into batches of input data;
a processing layer comprising:
one or more in-memory processing components positioned in communication with the one or more memory modules and configured to process the batches of input data in the one or more memory modules;
a plurality of LLMs configured to process the batches of input data and generate corresponding batches of processed output data; and
an output broker configured to:
cache results from the plurality of LLMs in a results cache;
assign priority scores to the cached results;
rank the cached results based on their priority scores; and
provide ranked results responsive to subsequent similar input data received by the data receiver from the results cache.

### Claim 2 (depends on 1)

The system of claim 1 wherein the one or more memory modules comprises at least one of random-access memory (RAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), magnetoresistive random-access memory (MRAM), or non-volatile random-access memory (NVRAM).

### Claim 3 (depends on 1)

The system of claim 1 wherein the output broker is further configured to:
detect similarity between subsequently received input data received by the data receiver and previously cached results;
retrieve relevant cached results based on the detected similarity;
update the cached results with new processed output data; and
maintain a priority ranking of cached results based on frequency of access.

### Claim 4 (depends on 1)

The system of claim 1 wherein the input data stream is received via an application programming interface (API).

### Claim 5 (depends on 4)

The system of claim 4 wherein the data receiver is implemented in a cloud container environment and is accessible via the API.

### Claim 6 (depends on 1)

The system of claim 1 wherein the output broker is implemented in a cloud container environment and is accessible via an API.

### Claim 7 (depends on 1)

The system of claim 1 wherein the one or more in-memory processing components are configured to process the batches of input data entirely in the one or more memory modules.

### Claim 8

A method for in-memory processing of large language model (LLM) operations, comprising:
receiving an input data stream at a data receiver;
determining an aggregation duration for collecting data from the input data stream;
aggregating input data from the input data stream over the determined aggregation duration;
breaking the aggregated input data into data batches sized for in-memory processing;
processing the data batches in one or more memory modules using multiple LLMs to generate processed output data;
caching the processed output data in a results cache by an output broker;
assigning priority scores by the output broker to the cached processed output data based on at least one of:
frequency of receiving similar input data;
computational cost of generating the output data; and
recency of the cached processed output data being added to the results cache; and
providing the cached processed output data to subsequently received similar input data based on the assigned priority scores.

### Claim 9 (depends on 8)

The method of claim 8 further comprising:
monitoring cache utilization;
removing the cached processed output data having a lower priority score from the cached results when cache utilization exceeds a predetermined threshold;
updating the cached results with new processed output data when subsequently received similar input data are processed; and
maintaining an index of cached results for rapid retrieval of cached processed output data responsive to input data similarity.

### Claim 10 (depends on 8)

The method of claim 8 further comprising:
detecting similarity by the output broker between subsequently received input data received by the data receiver and previously cached results;
retrieving by the output broker relevant cached results based on the detected similarity;
updating by the output broker the cached results with new processed output data; and
maintaining by the output broker a priority ranking of cached results based on frequency of access.

### Claim 11 (depends on 8)

The method of claim 8 wherein the data batches are processed entirely in the one or more memory modules.

### Claim 12 (depends on 8)

The method of claim 8 wherein processing the data batches in one or more memory modules comprises processing the data batches in one or more modules of random-access memory (RAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), magnetoresistive random-access memory (MRAM), or non-volatile random-access memory (NVRAM).

### Claim 13 (depends on 8)

The method of claim 8 wherein receiving the input data stream at the data receiver comprises receiving the input data stream at the data receiver via an application programming interface.

### Claim 14 (depends on 13)

The method of claim 13 further comprising implementing the data receiving in a cloud container environment.

### Claim 15 (depends on 8)

The method of claim 8 further comprising implementing the output broker in a cloud container environment.

### Claim 16

A system for processing large language model (LLM) operations, comprising:
one or more memory modules;
a non-transitory computer-readable storage medium having software stored thereon that is executable by a processor to:
operate a data receiver configured to:
receive an input data stream; and
break the input data stream into batches of input data;
operate a processing layer comprising:
one or more in-memory processing components positioned in communication with the one or more memory modules and configured to process the batches of input data in the one or more memory modules;
a plurality of LLMs configured to process the batches of input data and generate corresponding batches of processed output data; and
an output broker configured to:
cache results from the plurality of LLMs in a results cache;
assign priority scores to the cached results;
rank the cached results based on their priority scores; and
provide ranked results responsive to subsequent similar input data received by the data receiver from the results cache;
a processor coupled to the one or more memory modules and the storage medium and configured to execute the software stored on the storage medium; and
a communication device operable to send and receive data to and from the processor across a network.

### Claim 17 (depends on 16)

The system of claim 16 wherein the one or more memory modules comprises at least one of random-access memory (RAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), magnetoresistive random-access memory (MRAM), or non-volatile random-access memory (NVRAM).

### Claim 18 (depends on 16)

The system of claim 16 wherein the software is further configured to, when executed, operate the output broker to:
detect similarity between subsequently received input data received by the data receiver and previously cached results;
retrieve relevant cached results based on the detected similarity;
update the cached results with new processed output data; and
maintain a priority ranking of cached results based on frequency of access.

### Claim 19 (depends on 16)

The system of claim 16 wherein the software is further configured to, when executed, implement an application programming interface (API) to receive the input data stream.

### Claim 20 (depends on 19)

The system of claim 19 wherein the software is further configured to, when executed, implement the data receiver in a cloud container environment and to be accessible via the API.

### Claim 21 (depends on 16)

The system of claim 16 wherein the output broker is implemented in a cloud container environment and is accessible via an API.
