Systems and methods for improving cache efficiency and utilization

A method, performed by a graphics processing unit (GPU), the method comprising:

• processing data with a plurality of groups of cores, including:
• processing graphics data with a plurality of cores of a first type in each of the groups;
• performing matrix operations with a plurality of cores of a second type in each of the groups, wherein the plurality of cores of the second type are tensor cores;
• and
• storing data in a plurality of combined level 1 (L1) cache and shared memory units, each corresponding to a different group of cores of the plurality of groups of cores;
• sharing a level 2 (L2) cache by the plurality of groups of cores;
• accessing data from a memory by a plurality of memory controllers of the GPU;
• and
• performing a load instruction received from a first core of the plurality of groups of cores, including:
• selecting a cache control, based on the load instruction, out of a plurality of stored cache controls, wherein at least some of the cache controls have different cache eviction priorities;
• and
• applying the selected cache control to data allocated into the L2 cache.

A system comprising:

• a memory;
• and
• a graphics processing unit (GPU) coupled with the memory, the GPU comprising:
• a plurality of groups of cores, each group of cores including:
• a plurality of cores of a first type;
• and
• a plurality of cores of a second type, wherein the plurality of cores of the second type are tensor cores;
• a plurality of combined level 1 (L1) cache and shared memory units, each corresponding to a different group of cores of the plurality of groups of cores;
• a level 2 (L2) cache to be shared by the plurality of groups of cores;
• a plurality of memory controllers to couple the GPU to a memory;
• and
• cache controller circuitry associated with the L2 cache in response to a load instruction from a first core of the plurality of groups of cores, to:
• select a cache control, based on the load instruction, out of a plurality of stored cache controls, wherein at least some of the cache controls have different cache eviction priorities;
• and
• apply the selected cache control to data allocated into the L2 cache.