# US 2025/0094232 A1 · Programming Model for Resource-Constrained Scheduling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094232 A1
- Document ID: US20250094232A1
- Status: Published application
- Kind: A1
- Application number: 18/961,452
- Filed: 2024-11-27
- Published: 2025-03-20
- Earliest claimed: 2020-03-20
- Assignees: NVIDIA Corporation
- Applicants: Nvidia Corporation
- Inventors: Yury URALSKY; Henry MORETON; Matthijs de SMEDT; Lei YANG
- Web page: https://patentgrep.com/patent/US20250094232A1

## CPC

- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)}
- G06F 9/5011: {the resources being hardware resources other than CPUs, Servers and Terminals}
- G06F 9/5055: {considering software capabilities, i.e. software resources associated or available to the machine}
- G06F 9/5072: {Grid computing}
- G06F 9/52: Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 1/60: Memory management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present technology augments the GPU compute model to provide system-provided data marshalling characteristics of graphics pipelining to increase efficiency and reduce overhead. A simple scheduling model based on scalar counters (e.g., semaphores) abstract the availability of hardware resources. Resource releases can be done programmatically, and a system scheduler only needs to track the states of such counters/semaphores to make work launch decisions. Semantics of the counters/semaphores are defined by an application, which can use the counters/semaphores to represent the availability of free space in a memory buffer, the amount of cache pressure induced by the data flow in the network, or the presence of work items to be processed.

## Claims

### Claim 1

In a GPU comprising a non-transitory memory that stores software comprising instructions configured to configure the GPU to marshal data through a resource,
a hardware scheduler that autonomously launches nodes of a graph in response to monitoring occupancy of the resource, the hardware scheduler including an occupancy monitoring circuit comprising a free semaphore and a ready semaphore.

### Claim 2 (depends on 1)

The hardware scheduler of claim 1 wherein the occupancy monitoring circuit further comprises a hardware comparator that compares the free semaphore and the ready semaphore to predetermined values.

### Claim 3 (depends on 2)

The hardware scheduler of claim 2 wherein the predetermined values are zero.

### Claim 4 (depends on 1)

The hardware scheduler of claim 1 wherein the resource comprises an on-chip cache memory.

### Claim 5 (depends on 1)

The hardware scheduler of claim 1 wherein the occupancy monitoring circuit associates a first free semaphore with a first producer writing to a first field of a data record and associates a second free semaphore with a second producer writing to a second field of the data record, wherein the first free semaphore and the second free semaphore each refer to the same data record.

### Claim 6

A GPU comprising:
a plurality of GPU processing cores;
a non-transitory memory that stores instructions configured to configure the GPU processing cores to marshal data through a cache memory; and
a hardware scheduler that autonomously launches processing nodes defined by a graph in response to monitoring occupancy of the cache memory, the hardware scheduler including an occupancy monitoring circuit comprising a first circuit defining a free semaphore and a second circuit defining a ready semaphore.

### Claim 7 (depends on 6)

The GPU of claim 6 wherein the occupancy monitoring circuit further comprises a hardware comparator that compares the free semaphore and the ready semaphore to predetermined values.

### Claim 8 (depends on 7)

The GPU of claim 7 wherein the predetermined values are zero.

### Claim 9 (depends on 6)

The GPU of claim 6 wherein the resource comprises an on-chip cache memory.

### Claim 10 (depends on 6)

The GPU of claim 6 wherein the occupancy monitoring circuit is structured to associate a first free semaphore with a first producer writing to a first field of a data record and to associate a second free semaphore with a second producer writing to a second field of the data record, wherein the first free semaphore and the second free semaphore each refer to the same data record.

### Claim 11 (depends on 6)

The GPU of claim 6 wherein the GPU is configured to ignore a manner in which the cache memory fragments fixed sized block main memory allocations within the cache because scheduling decisions are not based on actual or worst case usage but rather on a total number of cache lines that have been reserved/acquired and released without regard for how those cache lines might map to physical memory addresses.

### Claim 12 (depends on 11)

The GPU of claim 11 wherein the hardware scheduler tracks physical memory usage based on a total number of cache lines being used for physical memory allocation without having to be concerned about which particular fixed sized memory blocks are or are not allocated.

### Claim 13 (depends on 12)

The GPU of claim 12 wherein the hardware scheduler does not track which cache blocks are or are not read or written, since the cache memory automatically tracks which cache blocks are or are not read or written.

### Claim 14 (depends on 6)

The GPU of claim 6 wherein the hardware scheduler ignores cache fragmentation by using existing cache allocation mechanisms to manage fragmentation while still allowing on-GPU data flow to remain in/through the cache memory.

### Claim 15 (depends on 14)

The GPU of claim 14 wherein the hardware scheduler manages a footprint of the cache memory in fixed sized chunks to get a benefit of on-chip data flow by leveraging ability of the cache memory to capture data flow that is sparse relative to a much larger memory allocation because the cache memory captures what is actually being read and written.