# US 2026/0099447 A1 · Prefetching portions of large language models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0099447 A1
- Document ID: US20260099447A1
- Status: Published application
- Kind: A1
- Application number: 19/326,681
- Filed: 2025-09-11
- Published: 2026-04-09
- Earliest claimed: 2024-10-04
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Usman SAJID; Marie Mai NGUYEN; Shuyi PEI; Younghoon KIM; Rekha PITCHUMANI
- Web page: https://patentgrep.com/patent/US20260099447A1

## CPC

- G06F 12/0862: with prefetch
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06N 3/063: using electronic means
- G06N 3/08: Learning methods
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Prefetching portions of large language models is disclosed. A token computed based on a user input to a generative large language model may be received. A portion of the generative large language model may be identified using the token and a machine learning model trained to identify portions of the generative large language model. The portion may be written into a memory. The generative large language model may be caused to generate an output based on the user input using the portion in the memory.

## Claims

### Claim 1

A method comprising:
receiving a token computed based on a user input to a generative large language model;
identifying a portion of the generative large language model using the token and a machine learning model trained to identify portions of the generative large language model;
writing the portion into a memory; and
causing the generative large language model to generate an output based on the user input using the portion in the memory.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the portion comprises a first subnetwork within a first layer of the generative large language model and a second subnetwork within a second layer of the generative large language model.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the portion further comprises a third subnetwork within the first layer of the generative large language model and a fourth subnetwork within the second layer of the generative large language model.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the first subnetwork comprises first weights and the third subnetwork comprises second weights that are independent of the first weights.

### Claim 5 (depends on 2)

The method according to claim 2, wherein the first subnetwork comprises a multi-layer perceptron.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the machine learning model is trained on training data describing input instances comprising a current token and at least one previous token and corresponding output instances comprising subnetworks within layers of the generative large language model.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the memory is included in at least one memory die attached to a base die.

### Claim 8

A system comprising:
a first memory;
a second memory; and
a processor coupled to the first memory and the second memory, the processor configured to:
receive a current token and at least one previous token generated based on a user input to a generative large language model;
identify subnetworks within the generative large language model by processing the current token and the at least one previous token using a machine learning model;
prefetch the subnetworks from the first memory into the second memory; and
cause the generative large language model to generate an output based on the user input using the subnetworks in the second memory.

### Claim 9 (depends on 8)

The system according to claim 8, wherein the subnetworks comprise a first subnetwork within a first layer of the generative large language model and a second subnetwork within the first layer of the generative large language model.

### Claim 10 (depends on 9)

The system according to claim 9, wherein the first layer of the generative large language model comprises a third subnetwork and a fourth subnetwork.

### Claim 11 (depends on 9)

The system according to claim 9, wherein first subnetwork comprises first weights and the second subnetwork comprises second weights that are independent of the first weights.

### Claim 12 (depends on 8)

The system according to claim 8, wherein the second memory is included in at least one memory die attached to a base die.

### Claim 13 (depends on 8)

The system according to claim 8, wherein the first memory comprises a low-power double data rate (LPDDR) memory.

### Claim 14 (depends on 13)

The system according to claim 13, wherein the first memory is connected to a LPDDR memory controller that is connected to a compute device.

### Claim 15 (depends on 8)

The system according to claim 8, wherein the generative large language model is configured to generate the output using the subnetworks in the second memory in a first amount of time and the generative large language model is configured to generate the output using the subnetworks in the first memory in a second amount of time that is greater than the first amount of time.

### Claim 16

A non-transitory computer-readable storage medium storing instructions that, responsive to execution by a processor, cause the processor to perform operations comprising:
receiving a token computed based on a user input to a generative large language model;
identifying portions of layers of the generative large language model by processing the token using a machine learning model;
writing the portions of the layers into a memory; and
generating an output based on the user input and the token with the generative large language model using the portions of the layers in the memory.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein a first portion of the portions of the layers comprises first weights and a second portion of the portions of the layers comprises second weights that are independent of the first weights.

### Claim 18 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein the portions of the layers comprise subnetworks of the layers.

### Claim 19 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein the machine learning model is trained on training data describing input instances comprising a current token and at least one previous token and corresponding output instances comprising subnetworks within the layers of the generative large language model.

### Claim 20 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein the memory is included in at least one memory die attached to a base die.
