# US 2025/0147766 A1 · Compiler algorithm for gpu prefetching

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0147766 A1
- Document ID: US20250147766A1
- Status: Published application
- Kind: A1
- Application number: 18/500,879
- Filed: 2023-11-02
- Published: 2025-05-08
- Earliest claimed: 2023-11-02
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Suryanarayana Murthy DURBHAKULA; Adarsh GOLIKERI; Siva Rama Krishna Reddy BOGI REDDY
- Web page: https://patentgrep.com/patent/US20250147766A1

## CPC

- G06F 9/3816: {Instruction alignment, e.g. cache line crossing}
- G06F 9/3877: {using a secondary processor, e.g. coprocessor (peripheral processor G06F13/12)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Aspects of the disclosure are directed to processor prefetching. In accordance with one aspect, processor prefetching includes determining if an exposed latency parameter is greater than zero; determining if a loop is present in a program code; determining if access overlap across fibers is true; and performing one of the following: a) insert a prefetch every quantity K fibers at a prefetch distance D; b) insert a prefetch every quantity K fibers with a maximum of quantity N iterations; or c) insert a prefetch per fiber with a maximum of quantity N iterations ahead; and wherein, the prefetch distance D is equal to an exposed latency parameter divided by an average instruction latency parameter, the quantity K is equal to a cache line size divided by a per fiber access size, and the quantity N is equal to an exposed latency parameter divided by an average iteration latency parameter.

## Claims

### Claim 1

An apparatus comprising:
a layer 1 (L1) cache memory unit;
a layer 2 (L2) cache memory unit coupled to the layer 1 (L1) cache memory unit, wherein the layer 2 (L2) cache memory unit has a slower memory access and a larger memory capacity than the layer 1 (L1) cache memory unit; and
a graphical processing unit (GPU) coupled to the layer 1 (L1) cache memory unit and the layer 2 (L2) cache memory unit, the GPU configured to perform the following:
a) determine if an exposed latency parameter is greater than zero;
b) determine if a loop is present in a program code; and
c) determine if access overlap across fibers is true.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the GPU is further configured to insert a prefetch every quantity K fibers at a prefetch distance D.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the prefetch distance D is equal to an exposed latency parameter divided by an average instruction latency parameter, and the quantity K is equal to a cache line size divided by a per fiber access size.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the GPU is further configured to insert prefetch every K fibers with a maximum of quantity N iterations.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the quantity N is equal to an exposed latency parameter divided by an average iteration latency parameter, and the quantity K is equal to a cache line size divided by a per fiber access size.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the GPU is further configured to insert prefetch per fiber with a maximum of quantity N iterations ahead.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the quantity N is equal to an exposed latency parameter divided by an average iteration latency parameter.

### Claim 8

A method comprising:
determining if an exposed latency parameter is greater than zero; and
determining if a loop is present in a program code.

### Claim 9 (depends on 8)

The method of claim 8 further comprising determining that the exposed latency parameter is greater than zero.

### Claim 10 (depends on 9)

The method of claim 9, further comprising determining if access overlap across fibers is true.

### Claim 11 (depends on 10)

The method of claim 10, wherein the loop is present.

### Claim 12 (depends on 11)

The method of claim 11, further comprising inserting a prefetch per fiber with a maximum of quantity N iterations ahead.

### Claim 13 (depends on 12)

The method of claim 12, wherein the access overlap across fibers is not true.

### Claim 14 (depends on 11)

The method of claim 11, wherein the access overlap across fibers is true.

### Claim 15 (depends on 14)

The method of claim 14, further comprising inserting a prefetch every quantity K fibers with a maximum of quantity N iterations.

### Claim 16 (depends on 15)

The method of claim 15, wherein the quantity N is equal to an exposed latency parameter divided by an average iteration latency parameter.

### Claim 17 (depends on 15)

The method of claim 15, wherein the quantity K is equal to a cache line size divided by a per fiber access size.

### Claim 18 (depends on 10)

The method of claim 10, wherein the loop is not present.

### Claim 19 (depends on 18)

The method of claim 18, further comprising determining if access overlap across fibers is true.

### Claim 20 (depends on 19)

The method of claim 19, wherein the access overlap across fibers is true.

### Claim 21 (depends on 20)

The method of claim 20, further comprising inserting a prefetch every quantity K fibers at a prefetch distance D.

### Claim 22 (depends on 21)

The method of claim 21, wherein the prefetch distance D is equal to an exposed latency parameter divided by an average instruction latency parameter.

### Claim 23 (depends on 22)

The method of claim 22, wherein the quantity K is equal to a cache line size divided by a per fiber access size.

### Claim 24 (depends on 19)

The method of claim 19, wherein the access overlap across fibers is not true.

### Claim 25 (depends on 24)

The method of claim 24, further comprising inserting a per fiber prefetch at a prefetch distance D.

### Claim 26 (depends on 25)

The method of claim 25, wherein the prefetch distance D is equal to an exposed latency parameter divided by an average instruction latency parameter.

### Claim 27

An apparatus comprising:
means for determining if an exposed latency parameter is greater than zero;
means for determining if a loop is present in a program code; and
means for determining if access overlap across fibers is true.

### Claim 28 (depends on 27)

The apparatus of claim 27, further comprising one of the following:
a) means for inserting a prefetch every quantity K fibers at a prefetch distance D wherein, the prefetch distance D is equal to an exposed latency parameter divided by the average instruction latency parameter and the quantity K is equal to a cache line size divided by a per fiber access size;
b) means for inserting a prefetch every quantity K fibers with a maximum of quantity N iterations, wherein the quantity N is equal to an exposed latency parameter divided by an average iteration latency parameter, and the quantity K is equal to a cache line size divided by a per fiber access size; or
c) means for inserting a prefetch per fiber with a maximum of quantity N iterations ahead, wherein the quantity N is equal to an exposed latency parameter divided by the average iteration latency parameter.

### Claim 29

A non-transitory computer-readable medium storing computer executable code, operable on a device comprising at least one processor and at least one memory coupled to the at least one processor, wherein the at least one processor is configured to implement a processor prefetching, the computer executable code comprising:
instructions for causing a computer to determine if an exposed latency parameter is greater than zero;
instructions for causing the computer to determine if a loop is present in a program code;
instructions for causing the computer to determine if access overlap across fibers is true; and
instructions for causing the computer to perform one of the following:
a) insert a prefetch every quantity K fibers at a prefetch distance D wherein, the prefetch distance D is equal to an exposed latency parameter divided by the average instruction latency parameter and the quantity K is equal to a cache line size divided by a per fiber access size;
b) insert a prefetch every quantity K fibers with a maximum of quantity N iterations, wherein the quantity N is equal to an exposed latency parameter divided by an average iteration latency parameter, and the quantity K is equal to a cache line size divided by a per fiber access size; or
c) insert a prefetch per fiber with a maximum of quantity N iterations ahead, wherein the quantity N is equal to an exposed latency parameter divided by the average iteration latency parameter.
