# US 2026/0073300 A1 · Execution of segmented machine learning models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073300 A1
- Document ID: US20260073300A1
- Status: Published application
- Kind: A1
- Application number: 19/303,258
- Filed: 2025-08-18
- Published: 2026-03-12
- Earliest claimed: 2020-06-19
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Umesh S. VAISHAMPAYAN; Gaurav KAPOOR; Kit-Man WAN
- Web page: https://patentgrep.com/patent/US20260073300A1

## CPC

- G06N 20/00: Machine learning
- G06F 9/5016: {the resource being the memory}
- G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)}
- G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30)
- G06F 2209/5017: Task decomposition
- G06F 2209/506: Constraint
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A device implementing a system to execute machine learning models from memory includes at least one processor configured to receive a request to provide an input to one or more machine learning (ML) models arranged into a graph of connected layers, the one or more ML models stored in the first type of memory. The at least one processor is further configured to divide the graph of connected layers into a plurality of segments such that at least two of the plurality of segments concurrently fits within allocated space of the second type of memory. The at least one processor is further configured to cause the input to be processed through the first segment of the plurality of segments using the second type of memory while a second segment of the plurality of segments is concurrently loaded from the first type of memory into the second type of memory.

## Claims

### Claim 1

A method, comprising:
receiving a request to provide an input to one or more machine learning (ML) models comprising layers, the one or more ML models stored in a first type of memory;
dividing layers of the one or more machine learning models into a plurality of segments such that at least one of the plurality of segments fits within allocated space of a second type of memory that differs from the first type of memory;
loading a first segment of the plurality of segments from the first type of memory into the second type of memory;
causing the input to be processed through the first segment of the plurality of segments using the second type of memory while a second segment of the plurality of segments is loaded from the first type of memory into the second type of memory; and
providing an output of the one or more machine learning models after performing the loading and processing for at least some of the plurality of segments.

### Claim 2 (depends on 1)

The method of claim 1, wherein dividing the layers is in response to determining that a total memory footprint of the layers exceeds the allocated space of the second type of memory.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first type of memory is a non-volatile memory, and the second type of memory is a volatile memory.

### Claim 4 (depends on 3)

The method of claim 3, wherein the first type of memory comprises NAND flash memory, and wherein the second type of memory comprises dynamic random access memory (DRAM).

### Claim 5 (depends on 1)

The method of claim 1, wherein the second type of memory includes a buffer cache for working data when processing the input through each of the plurality of segments.

### Claim 6 (depends on 1)

The method of claim 1, wherein dividing the layers into the plurality of segments is based on aligning to transitions between the one or more ML models.

### Claim 7 (depends on 1)

The method of claim 1, wherein dividing the layers into the plurality of segments is based on minimizing state information carried between the plurality of segments.

### Claim 8 (depends on 1)

The method of claim 1, wherein after causing the input to be processed through the first segment, the method further comprises:
loading a third segment of the plurality of segments from the first type of memory into the second type of memory to replace the first segment stored therein; and
causing the input to be processed through the second segment using the second type of memory while the third segment is concurrently loaded from the first type of memory into the second type of memory.

### Claim 9 (depends on 1)

The method of claim 1, wherein the plurality of segments are predetermined from stored metadata associated with the layers.

### Claim 10 (depends on 1)

The method of claim 1, wherein the layers is retrieved in an interleaved format optimized for parallel processing.

### Claim 11 (depends on 1)

The method of claim 1, wherein the output comprises at least one of a visual output and an audio output.

### Claim 12 (depends on 1)

The method of claim 1, wherein causing the input to be processed through the first segment comprises:
distributing parallelizable portions of the first segment for processing across multiple computing units of at least one of: one or more general purpose processors, one or more graphics processing units (GPUs), and one or more application specific integrated circuits (ASICs).

### Claim 13

A device, comprising:
a first type of memory;
a second type of memory that differs from the first type of memory; and
at least one processor configured to:
receive a request to provide an input to one or more machine learning (ML) models comprising layers, the one or more ML models stored in the first type of memory;
divide the layers of the one or more ML models into a plurality of segments such that at least one of the plurality of segments fits within allocated space of the second type of memory;
load a first segment of the plurality of segments from the first type of memory into the second type of memory;
cause the input to be processed through the first segment of the plurality of segments using the second type of memory while a second segment of the plurality of segments is loaded from the first type of memory into the second type of memory; and
provide an output of the one or more machine learning models after performing the loading and processing for at least some of the plurality of segments.

### Claim 14 (depends on 13)

The device of claim 13, wherein the first type of memory is a non-volatile memory, and the second type of memory is a volatile memory.

### Claim 15 (depends on 13)

The device of claim 13, wherein the at least one processor is configured to divide the layers into the plurality of segments based on aligning to transitions between the one or more ML models.

### Claim 16 (depends on 13)

The device of claim 13, wherein the at least one processor is configured to divide the layers into the plurality of segments based on minimizing state information carried between the plurality of segments.

### Claim 17 (depends on 13)

The device of claim 13, wherein after causing the input to be processed through the first segment, the at least one processor is further configured to:
load a third segment of the plurality of segments from the first type of memory into the second type of memory to replace the first segment stored therein; and
cause the input to be processed through the second segment using the second type of memory while the third segment is concurrently loaded from the first type of memory into the second type of memory.

### Claim 18 (depends on 13)

The device of claim 13, wherein the plurality of segments are predetermined from stored metadata associated with the layers.

### Claim 19

A computer program product comprising code, stored in a non-transitory computer-readable storage medium, the code comprising:
code to receive, by a first device, a request to provide an input to one or more machine learning (ML) models comprising layers, the one or more ML models stored in a first type of memory;
code to divide, by the first device, the layers of the one or more machine learning models into a plurality of segments such that at least one of the plurality of segments fits within allocated space of a second type of memory that differs from the first type of memory;
code to load, by the first device, a first segment of the plurality of segments from the first type of memory into the second type of memory;
code to cause, by the first device, the input to be processed through the first segment of the plurality of segments using the second type of memory while a second segment of the plurality of segments is loaded from the first type of memory into the second type of memory; and
code to provide, by the first device, an output of the one or more machine learning models after performing the loading and processing for at least some of the plurality of segments.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the code to divide, by the first device, the layers into the plurality of segments is based on at least one of: aligning to transitions between the one or more ML models, minimizing state information carried between the plurality of segments, and minimizing a size deviation between the plurality of segments.
