# US 2019/0377679 A1 · Extended line width memory-side cache systems and methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377679 A1
- Document ID: US20190377679A1
- Status: Published application
- Kind: A1
- Application number: 16/053,584
- Filed: 2018-08-02
- Published: 2019-12-12
- Earliest claimed: 2018-06-07
- Assignees: Micron Technology, Inc.
- Applicants: Micron Technology, Inc.
- Inventors: Richard C. Murphy; Anton Korzh; Stephen S. Pawlowski
- Web page: https://patentgrep.com/patent/US20190377679A1

## CPC

- G06F 3/0611: {in relation to response time}
- G06F 3/0634: {by changing the state or mode of one or more devices}
- G06F 3/0656: {Data buffering arrangements}
- G06F 3/0658: {Controller construction arrangements}
- G06F 12/0811: with multilevel cache hierarchies
- G06F 12/0815: Cache consistency protocols
- G06F 2212/1024: Latency reduction
- G06F 2212/1032: Reliability improvement, data loss prevention, degraded operation etc
- G06F 2212/601: Reconfiguration of cache memory
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present disclosure techniques for implementing an apparatus, which includes processing circuitry that performs an operation based a target data block, a processor-side cache that implements a first cache line, memory-side cache that implements a second cache line having line width greater than the first cache line, and a memory array. The apparatus includes one or more memory controllers that, when the target data block results in a cache miss, determine a row address that identifies a memory cell row as storing the target data block, instruct the memory array to successively output multiple data blocks from the memory cell row to enable the memory-side cache to store each of the multiple of data blocks in the second cache line, and instruct the memory-side cache to output the target data block to a coherency bus to enable the processing circuitry to perform the operation based on the target data block.

## Claims

### Claim 1

An apparatus comprising:
processing circuitry configured to perform a first operation based at least in part on a first target data block;
a processor-side cache coupled between the processing circuitry and a coherency bus, wherein the processor-side cache is configured to implement a first cache line with a first line width;
a memory-side cache coupled to the coherency bus, wherein the memory-side cache is configured to implement a second cache line with a second line width greater than the first line width implemented in the processor-side cache;
a memory array comprising a plurality of memory cell rows and a plurality of memory cell columns, wherein a set of memory cells located at each row address and column address pairing in the memory array is configured to store a data block; and
one or more memory controllers configured to, when the first target data block results in a cache miss:
determine a first row address that identifies a first memory cell row of the memory array as storing the first target data block, wherein the first memory cell row is configured to store a first plurality of data blocks;
instruct the memory array to successively output the first plurality of data blocks from the first memory cell row to enable the memory-side cache to store each of the first plurality of data blocks in the second cache line; and
instruct the memory-side cache to output the first target data block to the coherency bus to enable the processing circuitry to perform the first operation based at least in part on the first target data block.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the one or more memory controllers are configured to:
determine first spatio-temporal data access information associated with the first plurality of data blocks stored in the first memory cell row of the memory array and in the second cache line of the memory-side cache, wherein the first spatio-temporal data access information indicates when any of the first plurality of data blocks was most recently targeted by the processing circuitry, number of times any of the first plurality of data blocks have previously been targeted by the processing circuitry, or both; and
control data storage in the memory-side cache based at least in part on the first spatio-temporal data access information.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein:
the processor-side cache is configured to:
store the first target data block in the first cache line;
implement a third cache line with the first line width; and
store another data block in the third cache line; and
the one or more memory controllers are configured to:
determine first temporal data access information associated with the first target data block stored in the memory array and in the first cache line of the processor-side cache, wherein the first temporal data access information indicates parameters associated with a previous attempt by the processing circuitry to access the first target data block;
determine second temporal data access information associated with the other data block stored the third cache line of the processor-side cache, wherein the second temporal data access information indicates parameters associate with another previous attempt by the processing circuitry to access the other data block;
instruct the processor-side cache to flush the first cache line when the first temporal data access information and the second temporal data access indicate that the previous attempt to access the other data block is more recent than the previous attempt to access the first target data block; and
instruct the processor-side cache to flush the third cache line when the first temporal data access information and the second temporal data access information indicate that the previous attempt to access the first target data block is more recent than the previous attempt to access the other data block.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein:
a second memory cell row of the memory array is configured to store a second plurality of data blocks;
the memory-side cache comprises a third cache line configured to store each of the second plurality of data blocks; and
the one or more memory controllers are configured to:
determine second spatio-temporal data access information associated with the second plurality of data blocks stored in the second memory cell row of the memory array and in the third cache line of the memory-side cache, wherein the second spatio-temporal data access information indicates when any of the second plurality of data blocks was most recently targeted by the processing circuitry, number of times any of the second plurality of data blocks have previously been targeted by the processing circuitry, or both;
instruct the memory-side cache to flush the second cache line when the first spatio-temporal data access information and the second spatio-temporal data access information indicate that one or more of the second plurality of data blocks have been targeted by the processing circuitry more recently than any of the first plurality of data blocks, that the second plurality of data blocks have previously been targeted by the processing circuitry more times than the first plurality of data blocks, or both; and
instruct the memory-side cache to flush the third cache line when the first spatio-temporal data access information and the second spatio-temporal data access information indicate that one or more of the first plurality of data blocks have been targeted by the processing circuitry more recently than any of the second plurality of data blocks, that the first plurality of data blocks have previously been targeted by the processing circuitry more times than the second plurality of data blocks, or both.

### Claim 5 (depends on 2)

The apparatus of claim 2, wherein the one or more memory controllers are configured to update the first spatio-temporal data access information after each time the processing circuitry targets any of the first plurality of data blocks.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the memory-side cache is configured to:
store the first target data block in the second cache line before any other data blocks of the first plurality of data blocks; and
successively store each of the other data blocks of the first plurality of data blocks in the second cache line after the first target data block.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the memory-side cache is configured to:
output the second cache line including the first target data block to the coherency bus before one or more of the other data blocks of the first plurality of data blocks is stored in the second cache line; and
successively store one or more of the other data block of the first plurality of data blocks while the processing circuitry is performing the first operation based at least in part on the first target data block.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein:
the processing circuitry is configured to perform a second operation after the first operation based at least in part on a second target data block that is stored in the first memory cell row of the memory array along with the first target data block; and
the one or more memory controllers are configured to:
determine whether the second target data block is already stored in the second cache line of the memory-side cache; and
instruct the memory-side cache to output the second cache line to the coherency bus after the second target data block is stored in the second cache line to enable the processing circuitry to perform the second operation based at least in part on the second target data block.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein, when the first target data block results in the cache miss, the one or more memory controllers are configured to:
determine a first column address that identifies a first memory cell column that intersects the first memory cell row as storing the first target data block, wherein a second column address identifies a second memory cell column of the memory array that intersects the first memory cell row and stores another data block of the first plurality of data blocks;
instruct the memory array to output the first target data block by activating the first memory cell row based at least in part on the first row address and selecting the first memory cell column based at least in part on the first column address; and
instruct the memory array to output the other data block of the first plurality of data blocks by selecting the second memory cell column after the first memory cell column based at least in part on the second column address without deactivating the first memory cell row between selection of the first memory cell column and selection of the second memory cell column.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein:
the processing circuitry is configured to perform a second operation after the first operation based at least in part on a second target data block that is stored in the first memory cell row of the memory array along with the first target data block; and
the one or more memory controllers are configured to:
determine a third column address that identifies a third memory cell column that intersects the first memory cell row as storing the second target data block;
instruct the memory array to output the second target data block by selecting the third memory cell column based at least in part on the third column address, wherein the one or more memory controllers are configured to:
instruct the memory array to select the third memory cell column after the second memory cell column when the processing circuitry targets the second target data block after the second target data block is already stored in second cache line of the memory-side cache; and
instruct the memory array to select the third memory cell column before the second memory cell column when the processing circuitry targets the second target data block before the second target data block is stored in second cache line of the memory-side cache.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein:
the processor-side cache is configured to implement the first cache line such that the first line width is equal to size of data blocks expected to be requested by the processing circuitry; and
the memory-side cache is configured to implement the second cache line such that the second line width is equal to a page width of the memory array.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein:
the first target data block comprises a 64 byte data block or a 128 byte data block;
the processor-side cache is configured to implement the first cache line with a 64 byte line width when the first target data block comprises the 64 byte data block or with a 128 byte line width when the first target data block comprises the 128 byte data block;
the first memory cell row is configured to store one kilobyte of data, two kilobytes of data, four kilobytes of data, or eight kilobytes of data; and
the memory-side cache is configured to implement the second cache line with a one kilobyte line width when the first memory cell row is configured to store one kilobyte of data, with a two kilobyte line width when the first memory cell row is configured to store two kilobytes of data, with a four kilobyte line width when the first memory cell row is configured to store four kilobytes of data, or with an eight kilobyte line width when the first memory cell row is configured to store eight kilobytes of data.

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein the memory-side cache is configured to adaptively adjust the second line width of the second cache line based at least in part on an application that resulted in the processing circuitry targeting the first target data block, the first operation to be performed by the processing circuitry, or both.

### Claim 14 (depends on 1)

The apparatus of claim 1, wherein the one or more memory controllers are configured to:
store global metadata associated with the first plurality of data blocks in the second cache line of the memory-side cache;
store local metadata associated with each data block of the first plurality of data blocks in the second cache line of the memory-side cache; and
update the local metadata associated with a corresponding data block of the first plurality of data blocks to include a dirty indicator when the corresponding data block is modified after initially stored in the second cache line of the memory-side cache.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein, when a cache line flush instruction is received, the memory-side cache is configured to output only those of the first plurality of data blocks associated with local metadata that comprise the dirty indicator.

### Claim 16 (depends on 1)

The apparatus of claim 1, comprising:
a central processing unit, wherein the central processing unit comprises the processor-side cache integrated with the processing circuitry; and
one or more memory devices comprising a plurality of memory cells configured to be organized to implement the memory array.

### Claim 17 (depends on 1)

The apparatus of claim 1, wherein:
the processor-side cache comprises static random-access memory; and
the memory-side cache comprises dynamic random-access memory, static random-access memory, or a combination thereof.

### Claim 18

A method for controlling data storage in hierarchical caches implemented between a memory array and processing circuitry, comprising:
determining a row address and a column address that indicate storage location in the memory array of a first data block targeted by the processing circuitry;
outputting an activation signal from row select circuitry to a word line coupled to a first memory cell row of the memory array identified by the row address;
outputting a first column select signal to column select circuitry to enable the memory array to output the first data block from the first memory cell row to a first cache of the hierarchical caches via a first plurality of bit lines each coupled to a first memory cell column of the memory array identified by the column address;
outputting a first cache line comprising the first data block from the first cache to enable the processing circuitry to perform an operation based at least in part on the first data block; and
outputting a second column select signal to the column select circuitry to enable the memory array to output a second data block from the first memory cell row to the first cache via a second plurality of bit lines each coupled to a second memory cell column different from the first memory cell column while the processing circuitry is performing the operation based at least in part on the first data block.

### Claim 19 (depends on 18)

The method of claim 18, wherein:
outputting the first cache line from the first cache comprises outputting the first cache line from the first cache before the second data block is stored in the first cache line; and
outputting the second column select signal comprises outputting the second column select signal to the column select circuitry after the first cache outputs the first cache line.

### Claim 20 (depends on 18)

The method of claim 18, wherein outputting the first cache line from the first cache comprises outputting the first cache line from the first cache to a second cache of the hierarchical caches, wherein:
the first cache line is implemented in the first cache with a first cache line width; and
a second cache line is implemented in the second cache with a second cache line width smaller than the first cache line width.

### Claim 21 (depends on 18)

The method of claim 18, comprising adaptively adjusting line width of the first cache line implemented by the first cache based at least in part on an application that resulted in the processing circuitry targeting the first data block, the operation to be performed by the processing circuitry, or both.

### Claim 22 (depends on 18)

The method of claim 18, comprising re-outputting the first cache line from the first cache when the processing circuitry subsequently targets the second data block.

### Claim 23 (depends on 18)

The method of claim 18, comprising:
determining first spatio-temporal data access information associated with a first plurality of data blocks previously cached in the first cache line of the first cache, wherein each of the first plurality of data blocks is stored in a second memory cell row of the memory array and the first spatio-temporal data access information indicates a first most recent time any of the first plurality of data blocks was previously targeted by the processing circuitry;
determining second spatio-temporal data access information associated with a second plurality of data blocks previously cached in a second cache line of the first cache, wherein each of the second plurality of data blocks is stored in a third memory cell row of the memory array and the second spatio-temporal data access information indicates a second most recent time any of the second plurality of data blocks was previously targeted by the processing circuitry;
selecting the first cache line over the second cache line for caching each of a third plurality of data blocks stored in the first memory cell row of the memory array when the first spatio-temporal data access information and the second spatio-temporal data access information indicate that at least one of the first plurality of data blocks stored in the second memory cell row was targeted more recently than any of the second plurality of data blocks stored in the third memory cell row; and
updating third spatio-temporal data access information associated with the second data block stored in the first memory cell row to indicate when the first data block was targeted by the processing circuitry.

### Claim 24

A computing system comprising a memory sub-system, wherein the memory sub-system comprises:
one or more memory devices configured to be communicatively coupled to processing circuitry via one or more data buses, wherein the one or more memory devices comprise a first plurality of memory cells organized to form a first memory array corresponding with a first memory channel;
hierarchical memory caches coupled between the first memory array and the one or more data buses; and
a memory-side memory controller communicatively coupled to the first memory channel and the hierarchical memory caches, wherein the memory-side memory controller is configured to control caching in the hierarchical memory caches of first data blocks stored in the first memory array of the first memory channel based at least in part on a first spatial relationship between first storage locations of the first data blocks in the first memory array.

### Claim 25 (depends on 24)

The computing system of claim 24, comprising a processing sub-system communicatively coupled to the memory sub-system via the one or more data buses, wherein the processing sub-system comprises:
the processing circuitry, wherein the processing circuitry is configured to perform an operation based on a target data block;
hierarchical processor caches coupled between the processing circuitry and the one or more data buses; and
a processor-side memory controller communicatively coupled to the processing circuitry, the hierarchical processor caches, and the memory-side memory controller, wherein the processor-side memory controller is configured to control caching in the hierarchical memory caches of the first data blocks stored in the first memory array based at least in part on a temporal relationship between previous targeting of the first data blocks by the processing circuitry.

### Claim 26 (depends on 24)

The computing system of claim 24, wherein:
the memory sub-system comprises a second memory channel configured to store a second plurality of data blocks in a second memory array; and
the hierarchical memory caches comprise:
a first memory channel cache configured to cache one or more of the first plurality of data blocks stored in the first memory channel;
a second memory channel cache configured to cache one or more of the second plurality of data blocks stored in the second memory channel; and
a shared memory cache coupled the first memory channel cache and the second memory channel cache to provide the first memory channel cache and the second memory channel cache a lower cache level.

### Claim 27 (depends on 24)

The computing system of claim 24, wherein the hierarchical memory caches are configured to implement a cache line with a line width that is greater than or equal to an integer multiple of a size of a data block expected to be targeted by the processing circuitry.

### Claim 28 (depends on 24)

The computing system of claim 24, wherein the memory-side memory controller is configured to:
determine characteristics of an application running on the computing system that resulted in the processing circuitry requesting a target data block; and
adaptively instruct the hierarchical memory caches to adjust line width of a cache line based at least in part on the characteristics of the application running on the computing system.

### Claim 29 (depends on 24)

The computing system of claim 24, wherein:
the one or more memory devices comprise a second plurality of memory cells configured to be organized into a second memory array corresponding with a second memory channel; and
the memory-side memory controller is configured to control caching in the hierarchical memory caches of second data blocks stored in the second memory array of the second memory channel based at least in part on a second spatial relationship between second storage locations of the second data blocks in the second memory array.

### Claim 30 (depends on 24)

The computing system of claim 24, wherein the memory-side memory controller is configured to:
receive a memory access request from a processor-side memory controller that identifies a first data block targeted by the processing circuitry;
determine a row address and a column address that identifies storage location of the first data block in the first memory array of the first memory channel based at least in part on the memory access request; and
when the first data block results in a memory-side cache miss:
instruct the first memory channel to activate a memory cell row of the first memory array identified by the row address;
instruct the first memory channel to select a first memory cell column of the first memory array identified by the column address to cache the first data block in the hierarchical memory caches;
instruct the hierarchical memory caches to output a cache line with the first data block to the one or more data buses; and
after the cache line is output with the first data block, instruct the first memory channel to select a second memory cell column of the first memory array to cache a second data block stored in the memory cell row identified by the row address along with the first data block in the cache line.