# US 2026/0065411 A1 · Adaptive memory management system for efficient training of large language models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065411 A1
- Document ID: US20260065411A1
- Status: Published application
- Kind: A1
- Application number: 19/315,114
- Filed: 2025-08-29
- Published: 2026-03-05
- Earliest claimed: 2024-08-30
- Assignees: University of Massachusetts
- Applicants: University of Massachusetts
- Inventors: Hanmei Yang; Tongping Liu; Jin Zhou; Hui Guan
- Web page: https://patentgrep.com/patent/US20260065411A1

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 1/60: Memory management
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Various examples are provided related to adaptive memory management. In one example, a method includes determining a chunk size for organizing parameters, the chunk size determined based on execution order of parameters and selected to reduce memory waste; determining a swapping interval including one activation swapping block followed by an integer number of gradient checkpointing blocks, the swapping interval determined by dividing a time required to swap one transformer block by a computation time of the one transformer block; determining a number of persistent chunks and non-persistent chunks to offload model states from GPUs; determining a number of chunk buffers on the GPUs for prefetching and reusing the model states; determining a number of activation swapping blocks to offload activations from the GPUs; determining a number of transformer blocks to apply gradient checkpointing; and initiating a forward computation pass based upon the configurations. A backward computation pass can be performed.

## Claims

### Claim 1

A method for adaptive memory management, comprising:
determining a chunk size for organizing parameters, the chunk size determined based on execution order of parameters and selected to reduce memory waste;
determining a swapping interval comprising one activation swapping block followed by an integer number of gradient checkpointing blocks, the swapping interval determined by dividing a time required to swap one transformer block by a computation time of the one transformer block;
determining a number of persistent chunks and non-persistent chunks to offload model states from graphics processing units (GPUs);
determining a number of chunk buffers on the GPUs for prefetching and reusing the model states;
determining a number of activation swapping blocks to offload activations from the GPUs;
determining a number of transformer blocks to apply gradient checkpointing; and
initiating a forward computation pass based upon the swapping interval, the number of persistent chunks and non-persistent chunks to offload, the number of chunk buffers, the number of activation swapping blocks and the number of transformer blocks.

### Claim 2 (depends on 1)

The method of claim 1, wherein the forward computation pass comprises a plurality of swapping intervals thereby providing interleaved activation swapping and gradient checkpointing.

### Claim 3 (depends on 2)

The method of claim 2, wherein the forward computation pass is completed by a series of transformer blocks without optimization.

### Claim 4 (depends on 1)

The method of claim 1, further comprising performing a backward computation pass following the forward computation pass, the backward computation pass based upon the forward computation pass.

### Claim 5 (depends on 1)

The method of claim 1, comprising updating parameters of the model states on one or more of the GPUs, a central processing unit (CPU), or a combination of both.

### Claim 6 (depends on 1)

The method of claim 1, comprising determining a complete parameter chunk based on shards collected from the GPUs for determination of the number of persistent chunks.

### Claim 7 (depends on 6)

The method of claim 6, comprising uploading parameter chunks that are non-persistent chunks to the GPUs from a central processing unit (CPU) before collecting the shards.

### Claim 8 (depends on 7)

The method of claim 7, comprising:
storing computed gradients in the parameter chunks on the GPUs during the backward computation pass;
performing a gradient reduction by averaging computed gradients across the GPUs; and
offloading gradient chunks that are non-persistent chunks from the GPUs to the CPU after gradient reduction.

### Claim 9 (depends on 7)

The method of claim 7, wherein the parameter chunks comprise persistent chunks permanently storing model states on the GPUs and non-persistent chunks storing model states on the CPU with parameters uploaded to the GPUs.

### Claim 10 (depends on 1)

The method of claim 1, comprising monitoring memory changes and peak memory usage based upon hooks before and during operations.

### Claim 11 (depends on 10)

The method of claim 10, comprising predicting memory demand based at least in part upon monitored memory changes and peak memory usage.

### Claim 12 (depends on 10)

The method of claim 10, comprising providing an accurate runtime estimation for a given configuration based upon monitored memory changes and peak usage.

### Claim 13 (depends on 12)

The method of claim 12, wherein the hooks of hookable operations are monitored for memory changes before and during operations.

### Claim 14

A system for adaptive memory management, comprising:
a computing system comprising processing circuitry including a central processing unit (CPU), graphics processing units (GPUs), and memory;
adaptive memory management executable by the computing system, the adaptive memory management configured to, when executed by the computing system, at least:
determine a chunk size for organizing parameters, the chunk size determined based on execution order of parameters and selected to reduce memory waste;
determine a swapping interval comprising one activation swapping block followed by an integer number of gradient checkpointing blocks, the swapping interval determined by dividing a time required to swap one transformer block by a computation time of the one transformer block;
determine a number of persistent chunks and non-persistent chunks to offload model states from the GPUs;
determine a number of chunk buffers on the GPUs for prefetching and reusing the model states;
determine a number of activation swapping blocks to offload activations from the GPUs;
determine a number of transformer blocks to apply gradient checkpointing; and
initiate a forward computation pass based upon the swapping interval, the number of persistent chunks and non-persistent chunks to offload, the number of chunk buffers, the number of activation swapping blocks and the number of transformer blocks.

### Claim 15 (depends on 14)

The system of claim 14, wherein the forward computation pass comprises a plurality of swapping intervals thereby providing interleaved activation swapping and gradient checkpointing.

### Claim 16 (depends on 15)

The system of claim 15, wherein the forward computation pass is completed by a series of transformer blocks without optimization.

### Claim 17 (depends on 14)

The system of claim 14, wherein the adaptive memory management is further configured to perform a backward computation pass following the forward computation pass, the backward computation pass based upon the forward computation pass.

### Claim 18 (depends on 14)

The system of claim 14, wherein the adaptive memory management is further configured to update parameters of the model states on one or more of the GPUs, the CPU, or a combination of both.

### Claim 19 (depends on 14)

The system of claim 14, wherein the adaptive memory management is further configured to determine a complete parameter chunk based on shards collected from the GPUs for determination of the number of persistent chunks.

### Claim 20 (depends on 14)

The system of claim 14, wherein the adaptive memory management is further configured to monitor memory changes and peak memory usage based upon hooks before and during operations.
