US 2020/0401529 A1 · GPU CACHE MANAGEMENT BASED ON LOCALITY TYPE DETECTION

Abstract

Wavefront loading in a processor is managed and includes monitoring a selected wavefront of a set of wavefronts. Reuse of memory access requests for the selected wavefront is counted. A cache hit rate in one or more caches of the processor is determined based on the counted reuse. Based on the cache hit rate, subsequent memory requests of other wavefronts of the set of wavefronts are modified by including a type of reuse of cache lines in requests to the caches. In the caches, storage of data in the caches is based on the type of reuse indicated by the subsequent memory access requests. Reused cache lines are protected by preventing cache line contents from being replaced by another cache line for a duration of processing the set of wavefronts. Caches are bypassed when streaming access requests are made.

Claims20

Dependent claims

A method for managing wavefront loading in a processor, the method comprising:

monitoring a selected wavefront of a set of wavefronts, the selected wavefront executed by one or more compute units of a group of compute units of the processor by:
counting reuse of memory access requests for the selected wavefront;
and
calculating a cache hit rate of hits in one or more caches of the processor for instructions for the selected wavefront based on the counted reuse of the memory access requests;
and
based on the calculated cache hit rate, modifying subsequent memory requests of other wavefronts of the set of wavefronts by including a type of cache reuse in requests to at least one cache in the processor, wherein the type of cache reuse is based on an amount of cache reuse relative to a cache reuse threshold value.

3 more elements

A processing system comprising:

a central processing unit (CPU);
and
a group of parallel processing compute units coupled to the CPU and having memory caches, the CPU providing a kernel for the group of parallel processing compute units to execute, the group of parallel processing compute units comprising at least one compute unit that includes:
a monitor array having tag entries for identifying memory access requests within a wavefront of a set of wavefronts loaded into the group of parallel processing compute units for execution;
a locality detection table having a set of entries corresponding to a respective program counter of the memory access requests of the wavefront, wherein each entry of the locality detection table includes:
a reuse field for a count of a number of reuses of data made by the wavefront memory access requests;
and
a coalescer configured to consolidate memory access requests to the memory caches.

5 more elements

Dependencies are inferred from the claim text.

Gpu cache management based on locality type detection

Life of this patent

Gpu cache management based on locality type detection

Life of this patent

Abstract

Claims20