# US 2024/0004794 A1 · Cached system for managing state vectors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004794 A1
- Document ID: US20240004794A1
- Status: Published application
- Kind: A1
- Application number: 17/809,833
- Filed: 2022-06-29
- Published: 2024-01-04
- Earliest claimed: 2022-06-29
- Assignees: Xilinx, Inc.
- Applicants: Xilinx, Inc.
- Inventors: Noel J. Brady; Lars-Olof B Svensson
- Web page: https://patentgrep.com/patent/US20240004794A1

## CPC

- G06F 12/0815: Cache consistency protocols
- G06F 12/0842: for multiprocessing or multitasking
- G06F 9/3004: {to perform operations on memory}
- G06F 9/3851: {from multiple instruction streams, e.g. multistreaming}
- G06F 12/0857: {by multiple requestors}
- G06F 12/0875: with dedicated cache, e.g. instruction or stack
- G06F 2212/1008: Correctness of operation, e.g. memory ordering
- G06F 2212/1021: Hit rate improvement
- G06F 2212/1024: Latency reduction
- G06F 2212/454: Vector or matrix data
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A cache system includes a computational cache and a computational cache miss-handler. The computational cache is configured to cache state vectors and perform read-modify-write (RMW) operations on the cached state vectors responsive to received RMW commands. The computational cache miss-handler is configured to perform RMW operations on state vectors stored in a memory responsive to cache misses in the computational cache. The memory is external to the cache system.

## Claims

### Claim 1

A system, comprising:
a cache system including:
a computational cache configured to cache state vectors and perform read-modify-write (RMW) operations on the cached state vectors responsive to received RMW commands; and
a computational cache miss-handler configured to perform RMW operations on state vectors stored in a memory responsive to cache misses in the computational cache, wherein the memory is external to the cache system;
wherein the computational cache miss-handler includes a thread assignment circuit configured to assign the RMW commands corresponding to cache misses to threads and an RMW coalescing circuit configured to combine selected RMW commands belonging to a same thread and having a same index into a single RMW command.

### Claim 2 (depends on 1)

The system of claim 1, comprising:
a processor configured to submit the RMW commands to the cache system and continue performing one or more other operations without waiting for responses to the RMW commands.

### Claim 3 (depends on 1)

The system of claim 1, wherein the computational cache comprises:
an RMW circuit configured to receive the RMW commands; and
a computational cache memory configured to store the cached state vectors;
wherein the RMW circuit is configured to read, from the computational cache memory, the cached state vectors specified by the received RMW commands, modify at least a portion of the cached state vectors resulting in modified state vectors, and write the modified state vectors to the computational cache memory.

### Claim 4 (depends on 3)

The system of claim 3, wherein the computational cache comprises:
a mini-cache memory coupled to the RMW circuit;
wherein the RMW circuit is configured to process a plurality of the RMW commands directed to a same cached state vector and occurring within a specified time window using the mini-cache memory.

### Claim 5 (depends on 3)

The system of claim 3, wherein the computational cache miss-handler comprises:
an RMW execution circuit configured to, in response to the cache misses in the computational cache, read state vectors specified by the RMW commands from the memory, modify at least a portion of the state vectors resulting in modified state vectors, and write the modified state vectors to the memory.

### Claim 6 (depends on 5)

The system of claim 5, wherein, in response to the cache misses, the RMW circuit is configured to create versions of the state vectors in the computational cache memory, wherein the versions of the state vectors created in the computational cache memory, at least initially, are zero-valued state vectors.

### Claim 7 (depends on 6)

The system of claim 6, wherein, in response to a read command for a selected state vector,
the RMW execution circuit is configured to read the selected state vector from the memory; and
the RMW circuit is configured to combine the selected state vector read from the memory with a version of the selected state vector stored in the computational cache memory and output a result of the combining as the selected state vector.

### Claim 8

(canceled)

### Claim 9 (depends on 1)

The system of claim 1, wherein the computational cache miss-handler comprises:
an RMW execution circuit including a plurality of RMW command queues, wherein the RMW execution circuit is configured to store the RMW commands corresponding to the cache misses in the RMW command queues based on thread assignment;
wherein the RMW execution circuit is configured to execute the RMW commands from the plurality of RMW command queues using round-robin arbitration where different ones of the plurality of RMW command queues correspond to different memory banks of the memory.

### Claim 10 (depends on 9)

The system of claim 9, wherein the RMW execution circuit includes a single read command queue configured to store read commands, wherein the RMW execution circuit is configured to prioritize execution of a selected read command of a selected thread from the read queue over execution of a selected RMW command of the selected thread.

### Claim 11

A method, comprising:
receiving, within a cache system, a read-modify-write (RMW) command from a processor; and
in response to the RMW command, performing, using the cache system, an RMW operation on a state vector specified by the RMW command without further involvement of the processor, wherein the performing includes:
assigning the RMW command to a thread of a plurality of threads;
storing the RMW command in a selected RMW command queue of a plurality of RMW command queues based on thread assignment; and
executing the RMW command from the selected RMW command queue according to a round-robin arbitration technique where different ones of the plurality of RMW command queues correspond to different memory banks of a memory.

### Claim 12 (depends on 11)

The method of claim 11, wherein the processor is configured to submit the RMW command to the cache system and continue performing one or more other operations without waiting for a response to the RMW command.

### Claim 13 (depends on 11)

The method of claim 11, comprising:
receiving a further RMW command within a computational cache of the cache system;
determining that the state vector specified by the further RMW command is stored in a computational cache memory of the computational cache; and
reading, using an RMW circuit of the computational cache, from the computational cache memory, the state vector specified by the further RMW command, modifying at least a portion of the state vector resulting in a modified state vector, and writing the modified state vector to the computational cache memory.

### Claim 14 (depends on 13)

The method of claim 13, wherein the further RMW command is one of a plurality of further RMW commands specifying the state vector and received by the computational cache within a specified time window, the method comprising:
using a mini-cache memory coupled to the RMW circuit to process the plurality of further RMW commands.

### Claim 15 (depends on 11)

The method of claim 11, wherein the performing, using the cache system, the RMW operation on the state vector specified by the RMW command includes:
in response to determining that the state vector specified by the RMW command is not stored in a computational cache memory of the cache system, reading, using a computational cache miss-handler, the state vector specified by the RMW command from a memory, modifying at least a portion of the state vector resulting in a modified state vector, and writing the modified state vector to the memory.

### Claim 16 (depends on 15)

The method of claim 15, wherein, in response to determining that the state vector specified by the RMW command is not stored in the computational cache memory of the cache system, creating a version of the state vector in the computational cache memory, wherein the version of the state vector created in the computational cache memory is, at least initially, a zero-valued state vector.

### Claim 17 (depends on 11)

The method of claim 11, comprising, in response to a read command, received from the processor, specifying the state vector,
reading, by the cache system, the state vector specified by the read command from the memory;
combining, by the cache system, the state vector read from the memory with a version of the state vector stored in the cache system; and
outputting, by the cache system, a result of the combining as the state vector.

### Claim 18 (depends on 11)

The method of claim 11, comprising:
combining, by the cache system, the RMW command with at least one other RMW command belonging to a same thread and having a same index into a single RMW command.

### Claim 19

(canceled)

### Claim 20 (depends on 11)

The method of claim 11, comprising:
prioritizing execution of a read command of a selected thread over execution of the RMW command, wherein the RMW command is assigned the selected thread.

### Claim 21

A method, comprising:
receiving, within a cache system, a read-modify-write (RMW) command from a processor;
in response to determining that the RMW command is a cache miss, creating within a computational cache of the cache system a zero-valued state vector corresponding to a state vector specified by the RMW command that is stored in a memory and performing, using the cache system, an RMW operation on the state vector specified by the RMW command in the memory without further involvement of the processor;
updating the zero-valued state vector within the computational cache for a subsequent RMW operation specifying the state vector, wherein the state vector of the computational cache mismatches the corresponding state vector stored in the memory; and
in response to a read command or expulsion for the zero-valued state vector of the computational cache as updated, generating a combined state vector by combining the zero-valued state vector as updated from the computational cache with the corresponding state vector stored in the memory and outputting the combined state vector.

### Claim 22 (depends on 21)

The method of claim 21, wherein the RMW command is one of a plurality of RMW commands specifying the state vector and received within a specified window of time, the method comprising:
processing the plurality of RMW commands using a mini-cache memory of the cache system.
