# US 10,503,655 B2 · Data block sizing for channels in a multi-channel high-bandwidth memory > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,503,655 B2 - Document ID: US10503655B2 - Status: Granted - Kind: B2 - Application number: 15/216,524 - Filed: 2016-07-21 - Published: 2018-01-25 - Granted: 2019-12-10 - Earliest claimed: 2016-07-21 - Assignees: Advanced Micro Devices, Inc. - Applicants: Advanced Micro Devices, Inc. - Inventors: Mitesh R. Meswani; Jee Ho Ryoo - Web page: https://patentgrep.com/patent/US10503655B2 ## CPC - G06F 12/0893: Caches characterised by their organisation or structure - G06F 13/00: Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units (interface circuits for specific input/output devices G06F3/00 {; multiprogram control therefor G06F9/46}; multiprocessor systems G06F15/16 ) - G06F 2212/1024: Latency reduction - G06F 2212/60: Details of cache memory - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The described embodiments include a computing device that caches data acquired from a main memory in a high-bandwidth memory (HBM), the computing device including channels for accessing data stored in corresponding portions of the HBM. During operation, the computing device sets each of the channels so that data blocks stored in the corresponding portions of the HBM include corresponding numbers of cache lines. Based on records of accesses of cache lines in the HBM that were acquired from pages in the main memory, the computing device sets a data block size for each of the pages, the data block size being a number of cache lines. The computing device stores, in the HBM, data blocks acquired from each of the pages in the main memory using a channel having a data block size corresponding to the data block size for each of the pages. ## Claims ### Claim 1 A computing device, comprising: a processor; a main memory; a high-bandwidth memory (HBM) that is used as a cache by the processor for storing cache lines acquired from the main memory; and a plurality of channels coupled between the processor and the HBM, each channel comprising a set of signal routes for accessing data stored in a corresponding portion of the HBM; wherein the computing device is configured to: set, for each channel of the plurality of channels, a respective channel data block size, the channel data block size indicating a number of cache lines that are to be included in data blocks that are stored in a portion of the HBM that is accessed via that channel; set, for pages in the main memory, respective page data block sizes, the page data block size for each of the pages being set based on a record of accesses of cache lines stored in the HBM that were acquired from that page, and the page data block size being a number of cache lines; select a given channel from among the plurality of channels to be used for storing, in the HBM, data blocks acquired from each of the pages based on a relative size of the page data block size for that page and the channel data block size for each channel; and store, in the HBM, data blocks acquired from each of the pages using the given channel for that page. ### Claim 2 (depends on 1) The computing device of claim 1, wherein the computing device is further configured to: upon storing, in the HBM, a data block comprising cache lines acquired from a given page, monitor accesses of the data block, the monitoring comprising detecting which of the cache lines are accessed; and generate a record of accesses of cache lines, the record of accesses of cache lines associated with the given page and indicating cache lines that were accessed. ### Claim 3 (depends on 2) The computing device of claim 2, wherein the computing device is further configured to: store, in the HBM, the data block comprising cache lines acquired from the given page, the storing comprising: based on the indication of the cache lines that were accessed in the record of accesses of cache lines, acquiring, for the data block, cache lines from corresponding portions of the given page. ### Claim 4 (depends on 3) The computing device of claim 3, wherein the computing device is further configured to: write back cache lines from the data block stored in the HBM to the main memory, the writing back comprising: based on the indication of the cache lines that were accessed in the record of accesses of cache lines, writing back, from the data block, the cache lines into corresponding portions of the given page. ### Claim 5 (depends on 4) The computing device of claim 4, wherein the record of accesses of cache lines are bit vectors, with each position in each bit vector including a value indicating whether a corresponding cache line was accessed, and wherein the computing device further comprises: translation logic, the translation logic comprising circuit elements for generating, based on the bit vector, a mapping of the portions of pages from which cache lines are to be acquired and to which cache lines are to be written back. ### Claim 6 (depends on 2) The computing device of claim 2, wherein the computing device is further configured to: upon storing, in the HBM, the data block comprising cache lines acquired from the given page, reset an existing record of accesses of cache lines for the given page, thereby enabling updating the record of accesses of cache lines in the given page. ### Claim 7 (depends on 1) The computing device of claim 1, wherein setting, for each channel of the plurality of channels, the respective channel data block size comprises: at one or more times while the computing device is executing a workload: detecting, for the workload, a pattern of data accesses of cache lines in data blocks stored in the HBM; based on the pattern of data accesses, identifying an arrangement of cache line sizes for the workload; and based on the arrangement of cache line sizes, setting the respective channel data block size for each of the plurality of channels so that blocks of data stored in the corresponding portions of the HBM are of the corresponding sizes. ### Claim 8 (depends on 1) The computing device of claim 1, wherein setting, for each channel of the plurality of channels, the respective channel data block size comprises: setting the respective channel data block size for two or more of the plurality of channels so that data blocks stored in the corresponding portions of the HBM include different numbers of cache lines. ### Claim 9 (depends on 1) The computing device of claim 1, further comprising: a package, the processor and HBM being coupled together in the package, wherein the main memory is separate from and external to the package. ### Claim 10 A method for storing data in a high-bandwidth memory (HBM) used as a cache for data acquired from a main memory in a computing device, the computing device including a plurality of channels coupled between a processor and the HBM, each channel having a set of signal routes for accessing data stored in a corresponding portion of the HBM, the method comprising: setting, for each channel of the plurality of channels, a respective channel data block size, the channel data block size indicating a number of cache lines that are to be included in data blocks that are stored in a portion of the HBM that is accessed via that channel; setting, for pages in the main memory, respective page data block sizes, the page data block size for each of the pages being set based on a record of accesses of cache lines stored in the HBM that were acquired from that page, and the page data block size being a number of cache lines; selecting a given channel from among the plurality of channels to be used for storing, in the HBM, data blocks acquired from each of the pages based on a relative size of the page data block size for that page and the channel data block size for each channel; and storing, in the HBM, data blocks acquired from each of the pages using the given channel for that page. ### Claim 11 (depends on 10) The method of claim 10, further comprising: upon storing, in the HBM, a data block comprising cache lines acquired from a given page, monitoring accesses of the data block, the monitoring comprising detecting which of the cache lines are accessed; and generating a record of accesses of cache lines, the record of accesses of cache lines associated with the given page and indicating cache lines that were accessed. ### Claim 12 (depends on 11) The method of claim 11, further comprising: storing, in the HBM, the data block comprising cache lines acquired from the given page, the storing comprising: based on the indication of the cache lines that were accessed in the record of accesses of cache lines, acquiring, for the data block, cache lines from corresponding portions of the given page. ### Claim 13 (depends on 12) The method of claim 12, further comprising: writing back cache lines from the data block stored in the HBM to the main memory, the writing back comprising: based on the indication of the cache lines that were accessed in the record of accesses of cache lines, writing back, from the data block, the cache lines into corresponding portions of the given page. ### Claim 14 (depends on 13) The method of claim 13, wherein the record of accesses of cache lines are bit vectors, with each position in each bit vector including a value indicating whether a corresponding cache line was accessed, and wherein the method further comprises: generating, based on the bit vector, using translation logic, a mapping of the portions of pages from which cache lines are to be acquired and to which cache lines are to be written back. ### Claim 15 (depends on 11) The method of claim 11, further comprising: upon storing, in the HBM, the data block comprising cache lines acquired from the given page, resetting an existing record of accesses of cache lines for the given page, thereby enabling updating the record of accesses of cache lines in the given page. ### Claim 16 (depends on 10) The method of claim 10, wherein setting, for each channel of the plurality of channels, the respective channel data block size comprises: at one or more times while the computing device is executing a workload: detecting, for the workload, a pattern of data accesses of cache lines in data blocks stored in the HBM; based on the pattern of data accesses, identifying an arrangement of cache line sizes for the workload; and based on the arrangement of cache line sizes, setting the respective channel data block size for each of the plurality of channels so that blocks of data stored in the corresponding portions of the HBM are of the corresponding sizes. ### Claim 17 (depends on 10) The method of claim 10, wherein setting, for each channel of the plurality of channels, the respective channel data block size comprises: setting the respective channel data block size for two or more of the plurality of channels so that data blocks stored in the corresponding portions of the HBM include different numbers of cache lines.