Back to results
US 11,914,525 B2Granted
System cache optimizations for deep learning compute engines
AssigneeIntel Corporation
InventorsNeta Zmora, Eran Ben-Avi
Life of this patent
Estimating the term…
Abstract
In an example, an apparatus comprises a plurality of compute engines; and logic, at least partially including hardware logic, to detect a cache line conflict in a last-level cache (LLC) communicatively coupled to the plurality of compute engines; and implement context-based eviction policy to determine a cache way in the cache to evict in order to resolve the cache line conflict. Other embodiments are also disclosed and claimed.
Claims22
An apparatus comprising:
- deep learning (DL) hardware circuitry communicatively coupled to a last-level cache (LLC) by an interconnect, wherein the DL hardware circuitry to execute one or more layers of a DL network using the LLC;
- and
- a system cache controller communicably coupled to the DL hardware circuitry and the LLC, the system cache controller to:
- receive, from the DL hardware circuitry, a cache access request;
- responsive to a cache line conflict in the LLC for the cache access request, implement a context-based eviction policy to determine a cache way in the LLC to evict in order to resolve the cache line conflict;
- and
- responsive to a cache miss for the cache access request, select a variable cache line size to allocate and fill from the LLC, wherein the variable cache line size is selected based on a total size of data indicated in metadata received with the cache access request and based on data utilization of the LLC by the DL network.
4 more elements
A system comprising:
- a last-level cache (LLC);
- a processor having deep learning (DL) hardware circuitry communicatively coupled to the LLC by an interconnect, wherein the DL hardware circuitry execute one or more layers of a DL network using the LLC;
- and
- a system cache controller communicably coupled to the DL hardware circuitry and the LLC, the system cache controller to:
- receive, from the DL hardware circuitry, a cache access request;
- responsive to a cache line conflict in the LLC for the cache access request, implement a context-based eviction policy to determine a cache way in the LLC to evict in order to resolve the cache line conflict;
- and
- responsive to a cache miss for the cache access request, select a variable cache line size to allocate and fill from the LLC, wherein the variable cache line size is selected based on a total size of data indicated in metadata received with the cache access request and based on data utilization of the LLC by the DL network.
5 more elements
A method comprising:
- receive, from deep learning (DL) hardware circuitry communicatively coupled to a last-level cache (LLC) by an interconnect, a plurality of cache access requests;
- responsive to a cache line conflict in the LLC for a cache access request of the plurality of cache access requests, implementing a context-based eviction policy to determine a cache way in the LLC to evict in order to resolve the cache line conflict;
- and
- responsive to a cache miss for the cache access request, selecting a variable cache line size to allocate and fill from the LLC, wherein the variable cache line size is selected based on a total size of data indicated in metadata received with the cache access request and based on data utilization of the LLC by a DL network.
1 more element
A non-transitory computer-readable medium comprising one or more instructions that when executed on at least one processor configure the at least one processor to perform one or more operations to:
- receive, from a deep learning (DL) hardware circuitry communicatively coupled to a last-level cache (LLC) by an interconnect, a plurality of cache access requests;
- responsive to a cache line conflict in the LLC for a cache access request of the plurality of cache access requests, implementing a context-based eviction policy to determine a cache way in the LLC to evict in order to resolve the cache line conflict;
- and
- responsive to a cache miss for the cache access request, selecting a variable cache line size to allocate and fill from the LLC, wherein the variable cache line size is selected based on a total size of data indicated in metadata received with the cache access request and based on data utilization of the LLC by a DL network.
1 more element
Dependencies are inferred from the claim text.