# US 12,008,378 B2 · Mechanism for reducing coherence directory controller overhead for near-memory compute elements

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,378 B2
- Document ID: US12008378B2
- Status: Granted
- Kind: B2
- Application number: 18/132,879
- Filed: 2023-04-10
- Published: 2023-08-03
- Granted: 2024-06-11
- Earliest claimed: 2021-12-23
- Assignees: Advanced Micro Devices, Inc.
- Applicants: Advanced Micro Devices, Inc.
- Inventors: Varun Agrawal; Yasuko Eckert
- Web page: https://patentgrep.com/patent/US12008378B2

## CPC

- G06F 9/3895: {for complex operations, e.g. multidimensional or interleaved address generators, macros}
- G06F 9/30043: {LOAD or STORE instructions; Clear instruction}
- G06F 9/30105: {Register structure}
- G06F 9/30189: {according to execution mode, e.g. mode flag}
- G06F 12/0815: Cache consistency protocols
- G06F 12/0817: using directory methods
- G06F 12/0828: {with concurrent directory accessing, i.e. handling multiple concurrent coherency transactions}
- G06F 12/0891: using clearing, invalidating or resetting means
- G06F 13/1668: {Details of memory controller}
- G06F 15/7821: {Tightly coupled to memory, e.g. computational memory, smart memory, processor in memory}
- G06F 2212/1016: Performance improvement
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A parallel processing (PP) level coherence directory, also referred to as a Processing In-Memory Probe Filter (PimPF), is added to a coherence directory controller. When the coherence directory controller receives a broadcast PIM command from a host, or a PIM command that is directed to multiple memory banks in parallel, the PimPF accelerates processing of the PIM command by maintaining a directory for cache coherence that is separate from existing system level directories in the coherence directory controller. The PimPF maintains a directory according to address signatures that define the memory addresses affected by a broadcast PIM command. Two implementations are described: a lightweight implementation that accelerates PIM loads into registers, and a heavyweight implementation that accelerates both PIM loads into registers and PIM stores into memory.

## Claims

### Claim 1

An apparatus comprising a coherence directory controller configured to:
receive, from a processor, a parallel processing (PP) command for a plurality of memory addresses identified by a signature;
detect that a cache in the processor does not contain dirty data for each address of the plurality of memory addresses identified by the signature; and
send the PP command to a memory controller for parallel execution on a plurality of near-memory compute elements.

### Claim 2 (depends on 1)

The apparatus of claim 1 wherein the coherence directory controller is further configured to generate the plurality of memory addresses based on the signature.

### Claim 3 (depends on 1)

The apparatus of claim 1 wherein:
a memory address of the plurality of memory addresses contains a plurality of bits;
the signature contains a subset of the plurality of bits that the memory address contains.

### Claim 4 (depends on 1)

The apparatus of claim 1 wherein the coherence directory controller is further configured to issue a query, containing the signature, to each coherency directory of one or more PP level coherence directories.

### Claim 5 (depends on 1)

The apparatus of claim 1 wherein the coherence directory controller is further configured to identify, based on the signature, a plurality of memory banks.

### Claim 6 (depends on 1)

The apparatus of claim 1 wherein the coherence directory controller is further configured to store, into a PP level coherence directory, the signature.

### Claim 7 (depends on 6)

The apparatus of claim 6 wherein the coherence directory controller is further configured to update, in the PP level coherence directory, a bit in an entry that contains the signature.

### Claim 8 (depends on 6)

The apparatus of claim 6 wherein the PP level coherence directory contains an entry that contains: the signature and a respective status for each memory bank of a plurality of memory banks.

### Claim 9 (depends on 8)

The apparatus of claim 8 wherein the status for a memory bank of the plurality of memory banks consists of one or two bits.

### Claim 10 (depends on 1)

The apparatus of claim 1 wherein the processor is a graphics processing unit (GPU) or a core in a multicore-central processing unit (CPU).

### Claim 11

A system comprising:
a processor;
a plurality of near-memory compute elements;
a memory controller connected to the plurality of near-memory compute elements; and
a coherence directory controller, connected to the processor and the memory controller, and configured to:
receive, from the processor, a parallel processing (PP) command for a plurality of memory addresses identified by a signature;
detect that a cache in the processor does not contain dirty data for each address of the plurality of memory addresses identified by the signature; and
send, in response to the cache in the processor not containing dirty data for the plurality of memory addresses identified by the signature, the PP command to the memory controller for parallel execution on the plurality of near-memory compute elements.

### Claim 12 (depends on 11)

The system of claim 11 wherein the coherence directory controller is further configured to generate the plurality of memory addresses based on the signature.

### Claim 13 (depends on 11)

The system of claim 11 wherein:
a memory address of the plurality of memory addresses contains a plurality of bits;
the signature contains a subset of the plurality of bits that the memory address contains.

### Claim 14 (depends on 11)

The system of claim 11 wherein the coherence directory controller is further configured to issue a query, containing the signature, to each coherency directory of one or more PP level coherence directories.

### Claim 15 (depends on 11)

The system of claim 11 wherein the coherence directory controller is further configured to identify, based on the signature, a plurality of memory banks.

### Claim 16 (depends on 11)

The system of claim 11 wherein the coherence directory controller is further configured to store, into a PP level coherence directory, the signature.

### Claim 17 (depends on 16)

The system of claim 16 wherein the coherence directory controller is further configured to update, in the PP level coherence directory, a bit in an entry that contains the signature.

### Claim 18 (depends on 16)

The system of claim 16 wherein the PP level coherence directory contains an entry that contains: the signature and a respective status for each memory bank of a plurality of memory banks.

### Claim 19 (depends on 18)

The system of claim 18 wherein the status for a memory bank of the plurality of memory banks consists of one or two bits.

### Claim 20 (depends on 11)

The system of claim 11 wherein the processor is a graphics processing unit (GPU) or a core in a multicore-central processing unit (CPU).