System cache optimizations for deep learning compute engines
AssigneeIntel Corporation
InventorsNeta Zmora, Eran Ben-Avi
Life of this patent
Estimating the term…
Abstract
In an example, an apparatus comprises a plurality of compute engines; and logic, at least partially including hardware logic, to detect a cache line conflict in a last-level cache (LLC) communicatively coupled to the plurality of compute engines; and implement context-based eviction policy to determine a cache way in the cache to evict in order to resolve the cache line conflict. Other embodiments are also disclosed and claimed.
Claims20
A system comprising:
- a last level cache (LLC) dynamically divided into private caches each corresponding to compute engines performing concurrent compute operations on different deep learning (DL) layers of a DL neural network;
- DL hardware circuitry communicatively coupled to the LLC by an interconnect, wherein the DL hardware circuitry comprising the compute engines to execute the concurrent compute operations on the DL layers using the LLC, wherein each compute engine corresponds to a different one of the DL layers;
- and
- a system cache controller communicably coupled to the DL hardware circuitry and the LLC, the system cache controller to:
- receive, from the DL hardware circuitry, a cache access request from a first compute engine of the compute engines performing the concurrent compute operations for a first DL layer of the DL neural network;
- and
- direct the cache access request to a first private cache of the private caches of the LLC, the first private cache corresponding to the first compute engine.
A method comprising:
- dividing a last level cache (LLC) into private caches each corresponding to compute engines performing concurrent compute operations on different deep learning (DL) layers of a DL neural network;
- executing, by DL hardware circuitry communicatively coupled to the LLC via an interconnect, the concurrent compute operations on the DL layers using the LLC, wherein the DL hardware circuitry comprises the compute engines to perform the concurrent compute operations, and wherein each compute engine corresponds to a different one of the DL layers;
- receiving, by a system cache controller from the DL hardware circuitry, a cache access request from a first compute engine of the compute engines performing the concurrent compute operations for a first DL layer of the DL neural network;
- and
- directing, by the system cache controller, the cache access request to a first private cache of the private caches of the LLC, the first private cache corresponding to the first compute engine.
A non-transitory computer-readable medium comprising one or more instructions that when executed on at least one processor configure the at least one processor to perform one or more operations to:
- dividing a last level cache (LLC) into private caches each corresponding to compute engines performing concurrent compute operations on different deep learning (DL) layers of a DL neural network;
- executing, by DL hardware circuitry communicatively coupled to the LLC via an interconnect, the concurrent compute operations on the DL layers using the LLC, wherein the DL hardware circuitry comprises the compute engines to perform the concurrent compute operations, and wherein each compute engine corresponds to a different one of the DL layers;
- receiving, by a system cache controller from the DL hardware circuitry, a cache access request from a first compute engine of the compute engines performing the concurrent compute operations for a first DL layer of the DL neural network;
- and
- directing, by the system cache controller, the cache access request to a first private cache of the private caches of the LLC, the first private cache corresponding to the first compute engine.
Dependencies are inferred from the claim text.