# US 2018/0024935 A1 · Data Block Sizing for Channels in a Multi-Channel High-Bandwidth Memory

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0024935 A1
- Document ID: US20180024935A1
- Status: Published application
- Kind: A1
- Application number: 15/216,524
- Filed: 2016-07-21
- Published: 2018-01-25
- Earliest claimed: 2016-07-21
- Assignees: Advanced Micro Devices, Inc.
- Applicants: Advanced Micro Devices, Inc.
- Inventors: Mitesh R. Meswani; Jee Ho Ryoo
- Web page: https://patentgrep.com/patent/US20180024935A1

## CPC

- G06F 12/0893: Caches characterised by their organisation or structure
- G06F 13/00: Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units (interface circuits for specific input/output devices G06F3/00 {; multiprogram control therefor G06F9/46}; multiprocessor systems G06F15/16 )
- G06F 2212/1024: Latency reduction
- G06F 2212/60: Details of cache memory
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The described embodiments include a computing device that caches data acquired from a main memory in a high-bandwidth memory (HBM), the computing device including channels for accessing data stored in corresponding portions of the HBM. During operation, the computing device sets each of the channels so that data blocks stored in the corresponding portions of the HBM include corresponding numbers of cache lines. Based on records of accesses of cache lines in the HBM that were acquired from pages in the main memory, the computing device sets a data block size for each of the pages, the data block size being a number of cache lines. The computing device stores, in the HBM, data blocks acquired from each of the pages in the main memory using a channel having a data block size corresponding to the data block size for each of the pages.

## Claims

### Claim 1

A computing device, comprising:
a processor;
a main memory;
a high-bandwidth memory (HBM) that is used as a cache by the processor for storing cache lines acquired from the main memory; and
a plurality of channels coupled between the processor and the HBM, each channel comprising a set of signal routes for accessing data stored in a corresponding portion of the HBM;
wherein the computing device is configured to:
set each of the plurality of channels so that data blocks stored in the corresponding portions of the HBM include corresponding numbers of cache lines;
based on records of accesses of cache lines in the HBM that were acquired from pages in the main memory, set a data block size for each of the pages, the data block size being a number of cache lines; and
store, in the HBM, data blocks acquired from each of the pages in the main memory using a channel having a data block size corresponding to the data block size for each of the pages.

### Claim 2 (depends on 1)

The computing device of claim 1, wherein the computing device is further configured to:
upon storing, in the HBM, a data block comprising cache lines acquired from a given page, monitor accesses of the data block, the monitoring comprising detecting which of the cache lines are accessed; and
generate a record of the data accesses, the record associated with the given page and indicating cache lines that were accessed.

### Claim 3 (depends on 2)

The computing device of claim 2, wherein the computing device is further configured to:
store, in the HBM, the data block comprising cache lines acquired from the given page, the storing comprising:
based on the indication of the cache lines that were accessed in the record of the data accesses, acquiring, for the data block, cache lines from corresponding portions of the given page.

### Claim 4 (depends on 3)

The computing device of claim 3, wherein the computing device is further configured to:
write back cache lines from the data block stored in the HBM to the main memory, the writing back comprising:
based on the indication of the cache lines that were accessed in the record of the data accesses, writing back, from the data block, the cache lines into corresponding portions of the given page.

### Claim 5 (depends on 4)

The computing device of claim 4, wherein the records of data accesses are bit vectors, with each position in each bit vector including a value indicating whether a corresponding cache line was accessed, and wherein the computing device further comprises:
translation logic, the translation logic comprising circuit elements for generating, based on the bit vector, a mapping of the portions of pages from which cache lines are to be acquired and to which cache lines are to be written back.

### Claim 6 (depends on 2)

The computing device of claim 2, wherein the computing device is further configured to:
upon storing, in the HBM, the data block comprising cache lines acquired from the given page, reset an existing record of data accesses for the given page, thereby enabling updating the records of data accesses in the pages.

### Claim 7 (depends on 1)

The computing device of claim 1, wherein setting each of the plurality of channels comprises:
at one or more times while the computing device is executing a workload:
detecting, for the workload, a pattern of data accesses of cache lines in data blocks stored in the HBM;
based on the pattern of data accesses, identifying an arrangement of cache line sizes for the workload; and
based on the arrangement of cache line sizes, setting each of the plurality of channels so that blocks of data stored in the corresponding portions of the HBM are of the corresponding sizes.

### Claim 8 (depends on 1)

The computing device of claim 1, wherein setting each of the plurality of channels comprises:
setting two or more of the plurality of channels so that data blocks stored in the corresponding portions of the HBM include different numbers of cache lines.

### Claim 9 (depends on 1)

The computing device of claim 1, further comprising:
a package, the processor and HBM being coupled together in the package, wherein the main memory is separate from and external to the package.

### Claim 10

A method for storing data in a high-bandwidth memory (HBM) used as a cache for data acquired from a main memory in a computing device, the computing device including a plurality of channels coupled between a processor and the HBM, each channel having a set of signal routes for accessing data stored in a corresponding portion of the HBM, the method comprising:
setting each of the plurality of channels so that data blocks stored in the corresponding portions of the HBM include corresponding numbers of cache lines;
based on records of accesses of cache lines in the HBM that were acquired from pages in the main memory, setting a data block size for each of the pages, the data block size being a number of cache lines; and
storing, in the HBM, data blocks acquired from each of the pages in the main memory using a channel having a data block size corresponding to the data block size for each of the pages.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
upon storing, in the HBM, a data block comprising cache lines acquired from a given page, monitoring accesses of the data block, the monitoring comprising detecting which of the cache lines are accessed; and
generating a record of the data accesses, the record associated with the given page and indicating cache lines that were accessed.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
storing, in the HBM, the data block comprising cache lines acquired from the given page, the storing comprising:
based on the indication of the cache lines that were accessed in the record of the data accesses, acquiring, for the data block, cache lines from corresponding portions of the given page.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
writing back cache lines from the data block stored in the HBM to the main memory, the writing back comprising:
based on the indication of the cache lines that were accessed in the record of the data accesses, writing back, from the data block, the cache lines into corresponding portions of the given page.

### Claim 14 (depends on 13)

The method of claim 13, wherein the records of data accesses are bit vectors, with each position in each bit vector including a value indicating whether a corresponding cache line was accessed, and wherein the method further comprises:
generating, based on the bit vector, using translation logic, a mapping of the portions of pages from which cache lines are to be acquired and to which cache lines are to be written back.

### Claim 15 (depends on 11)

The method of claim 11, further comprising:
upon storing, in the HBM, the data block comprising cache lines acquired from the given page, resetting an existing record of data accesses for the given page, thereby enabling updating the records of data accesses in the pages.

### Claim 16 (depends on 10)

The method of claim 10, wherein setting each of the plurality of channels comprises:
at one or more times while the computing device is executing a workload:
detecting, for the workload, a pattern of data accesses of cache lines in data blocks stored in the HBM;
based on the pattern of data accesses, identifying an arrangement of cache line sizes for the workload; and
based on the arrangement of cache line sizes, setting each of the plurality of channels so that blocks of data stored in the corresponding portions of the HBM are of the corresponding sizes.

### Claim 17 (depends on 10)

The method of claim 10, wherein setting each of the plurality of channels comprises:
setting two or more of the plurality of channels so that data blocks stored in the corresponding portions of the HBM include different numbers of cache lines.

### Claim 18

A translation logic circuit, comprising:
a bitmask storage element, the bitmask storage element comprising a plurality of bit positions;
a plurality of ripple carry adders, a first input of each adder coupled to an output of a previous adder or an initial input, and a second input of each adder coupled to a corresponding bit position in the bitmask storage element; and
offset selection logic, the offset selection logic configured to receive outputs of the ripple carry adders and to generate therefrom a value representing a location of a cache line in a data block in a high-bandwidth memory.

### Claim 19 (depends on 18)

The translation logic of claim 18, further comprising:
a plurality of tri-state buffers, an input of each tri-state buffer coupled to an output of a corresponding ripple carry adder, and a tri-state input of each tri-state buffer coupled to a corresponding bit position in the bitmask storage element; and
offset matching logic, the offset matching logic configured to receive outputs of the tri-state buffers and to generate therefrom a value representing a location of a cache line in a main memory.