# US 12,182,062 B1 · Multi-tile memory management

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,062 B1
- Document ID: US12182062B1
- Status: Granted
- Kind: B1
- Application number: 17/961,833
- Filed: 2022-10-07
- Granted: 2024-12-31
- Earliest claimed: 2019-03-15
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Abhishek R. Appu; Altug Koker; Aravindh Anantaraman; Elmoustapha Ould-Ahmed-Vall; Valentin Andrei; Nicolas Galoppo Von Borries; Varghese George; Mike Macpherson; Subramaniam Maiyuran; Joydeep Ray; Lakshminarayanan Striramassarma; Scott Janus; Brent Insko; Vasanth Ranganathan; Kamal Sinha; Arthur Hunter; Prasoonkumar Surti; David Puffer; James Valerio; Ankur N. Shah
- Web page: https://patentgrep.com/patent/US12182062B1

## CPC

- G06F 15/7839: {with memory}
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 7/575: Basic arithmetic logic units, i.e. devices selectable to perform either addition, subtraction or one of several logical operations, using, at least partially, the same circuitry
- G06F 7/58: Random or pseudo-random number generators
- G06F 7/588: {Random number generators, i.e. based on natural stochastic processes}
- G06F 9/3001: {Arithmetic instructions}
- G06F 9/30014: {with variable precision}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/3004: {to perform operations on memory}
- G06F 9/30043: {LOAD or STORE instructions; Clear instruction}
- G06F 9/30047: {Prefetch instructions; cache control instructions}
- G06F 9/30065: {Loop control instructions; iterative instructions, e.g. LOOP, REPEAT}
- G06F 9/30079: {Pipeline control instructions, e.g. multicycle NOP}
- G06F 9/383: {Operand prefetching (cache prefetching G06F12/0862)}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 9/3888: {controlled by a single instruction for multiple threads [SIMT] in parallel}
- G06F 9/5011: {the resources being hardware resources other than CPUs, Servers and Terminals}
- G06F 9/5066: {Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451)}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 12/0215: {with look ahead addressing means}
- G06F 12/0238: {Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory}
- G06F 12/0246: {in block erasable memory, e.g. flash memory}
- G06F 12/0607: {Interleaved addressing}
- G06F 12/0802: Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F 12/0804: with main memory updating (G06F12/0806 takes precedence)
- G06F 12/0811: with multilevel cache hierarchies
- G06F 12/0862: with prefetch
- G06F 12/0866: for peripheral storage systems, e.g. disk cache
- G06F 12/0871: Allocation or management of cache space
- G06F 12/0875: with dedicated cache, e.g. instruction or stack
- G06F 12/0882: Page mode
- G06F 12/0888: using selective caching, e.g. bypass
- G06F 12/0891: using clearing, invalidating or resetting means
- G06F 12/0893: Caches characterised by their organisation or structure
- G06F 12/0895: of parts of caches, e.g. directory or tag array
- G06F 12/0897: with two or more cache hierarchy levels (with multilevel cache hierarchies G06F12/0811)
- G06F 12/1009: using page tables, e.g. page table structures
- G06F 12/128: adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel
- G06F 13/1626: {by reordering requests}
- G06F 15/173: using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
- G06F 15/8046: {Systolic arrays}
- G06F 16/24569: {Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 1/60: Memory management
- H03M 7/46: Conversion to or from run-length codes, i.e. by representing the number of consecutive digits, or groups of digits, of the same kind by a code word and a digit indicative of that kind
- G06F 9/3802: {Instruction prefetching}
- G06F 9/3818: {Decoding for concurrent execution}
- G06F 9/3867: {using instruction pipelines}
- G06F 12/12: Replacement control
- G06F 2212/1008: Correctness of operation, e.g. memory ordering
- G06F 2212/1016: Performance improvement
- G06F 2212/1021: Hit rate improvement
- G06F 2212/1024: Latency reduction
- G06F 2212/1044: Space efficiency improvement
- G06F 2212/2542: Non-uniform memory access [NUMA] architecture
- G06F 2212/302: In image processor or graphics adapter
- G06F 2212/401: Compressed data
- G06F 2212/455: Image or video data
- G06F 2212/60: Details of cache memory
- G06F 2212/601: Reconfiguration of cache memory
- G06F 2212/6026: Prefetching based on access pattern detection, e.g. stride based prefetch
- G06F 2212/6028: Prefetching based on hints or prefetch instructions
- G06F 2212/608: Details relating to cache mapping
- G06F 2212/652: Page size control
- G06N 3/08: Learning methods
- G06T 15/06: Ray-tracing
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)

## Abstract

Methods and apparatus relating to techniques for multi-tile memory management. In an example, a graphics processor includes an interposer, a first chiplet coupled with the interposer, the first chiplet including a graphics processing resource and an interconnect network coupled with the graphics processing resource, cache circuitry coupled with the graphics processing resource via the interconnect network, and a second chiplet coupled with the first chiplet via the interposer, the second chiplet including a memory-side cache and a memory controller coupled with the memory-side cache. The memory controller is configured to enable access to a high-bandwidth memory (HBM) device, the memory-side cache is configured to cache data associated with a memory access performed via the memory controller, and the cache circuitry is logically positioned between the graphics processing resource and a chiplet interface.

## Claims

### Claim 1

A graphics processor comprising:
an interposer;
a first chiplet coupled with the interposer, the first chiplet including a graphics processing resource and a switched interconnect network coupled with the graphics processing resource;
cache circuitry coupled with the graphics processing resource via the switched interconnect network; and
a second chiplet coupled with the first chiplet via the interposer, the second chiplet including a memory-side cache and a memory controller coupled with the memory-side cache,
wherein the memory controller is configured to enable access to a high-bandwidth memory (HBM) device,
the memory-side cache is configured to cache data associated with a memory access performed via the memory controller, and
the cache circuitry is logically positioned between the graphics processing resource and a chiplet interface, wherein the first chiplet and the second chiplet are distinct and at least partially packaged integrated circuits.

### Claim 2 (depends on 1)

The graphics processor of claim 1, wherein the cache circuitry includes a level-2 (L2) cache.

### Claim 3 (depends on 2)

The graphics processor of claim 2, wherein the L2 cache is a distributed cache having nodes interconnected via the switched interconnect network.

### Claim 4 (depends on 1)

The graphics processor of claim 1, wherein the memory-side cache includes a level-3 (L3) cache.

### Claim 5 (depends on 1)

The graphics processor of claim 1, wherein the interposer, first chiplet, and second chiplet have a 2.5 dimension (2.5D) arrangement and the interposer includes a first standardized slot configured to accept the first chiplet and a second standardized slot configured to accept the second chiplet.

### Claim 6 (depends on 5)

The graphics processor of claim 5, wherein the interposer is an active interposer.

### Claim 7 (depends on 1)

The graphics processor of claim 1, wherein the graphics processing resource includes a plurality of functional units having a single instruction multiple thread (SIMT) architecture.

### Claim 8 (depends on 7)

The graphics processor of claim 7, wherein the plurality of functional units includes a general-purpose graphics processor core and a tensor core.

### Claim 9 (depends on 7)

The graphics processor of claim 7, wherein the plurality of functional units includes a ray-tracing core.

### Claim 10 (depends on 1)

The graphics processor of claim 1, wherein the graphics processing resource includes a plurality of functional units having a single instruction multiple data (SIMD) architecture.

### Claim 11

A system comprising:
an interconnect to a system interface; and
a multi-die graphics processor coupled with the interconnect, the multi-die graphics processor comprising:
an interposer;
a first chiplet coupled with the interposer, the first chiplet including a graphics processing resource and a switched interconnect network coupled with the graphics processing resource;
cache circuitry coupled with the graphics processing resource via the switched interconnect network; and
a second chiplet coupled with the first chiplet via the interposer, the second chiplet including a memory-side cache and a memory controller coupled with the memory-side cache, wherein the memory controller is configured to enable access to a high-bandwidth memory (HBM) device, the memory-side cache is configured to cache data associated with a memory access performed via the memory controller, and the cache circuitry is logically positioned between the graphics processing resource and a chiplet interface, wherein the first chiplet and the second chiplet are distinct and at least partially packaged integrated circuits.

### Claim 12 (depends on 11)

The system of claim 11, wherein the cache circuitry includes a level-2 (L2) cache.

### Claim 13 (depends on 12)

The system of claim 12, wherein the L2 cache is a distributed cache having nodes interconnected via the switched interconnect network.

### Claim 14 (depends on 11)

The system of claim 11, wherein the memory-side cache includes a level-3 (L3) cache.

### Claim 15 (depends on 11)

The system of claim 11, wherein the interposer, first chiplet, and second chiplet have a 2.5 dimension (2.5D) arrangement and the interposer includes a first standardized slot configured to accept the first chiplet and a second standardized slot configured to accept the second chiplet.

### Claim 16 (depends on 15)

The system of claim 15, wherein the interposer is an active interposer.

### Claim 17 (depends on 11)

The system of claim 11, wherein the graphics processing resource includes a plurality of functional units having a single instruction multiple thread (SIMT) architecture.

### Claim 18 (depends on 17)

The system of claim 17, wherein the plurality of functional units includes a general-purpose system core and a tensor core.

### Claim 19 (depends on 17)

The system of claim 17, wherein the plurality of functional units includes a ray-tracing core.

### Claim 20 (depends on 11)

The system of claim 11, wherein the graphics processing resource includes a plurality of functional units having a single instruction multiple data (SIMD) architecture.