# US 12,182,024 B2 · Latency reduction using stream cache > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,024 B2 - Document ID: US12182024B2 - Status: Granted - Kind: B2 - Application number: 18/508,141 - Filed: 2023-11-13 - Published: 2024-03-14 - Granted: 2024-12-31 - Earliest claimed: 2021-12-21 - Assignees: Micron Technology, Inc. - Applicants: Micron Technology, Inc. - Inventors: Muthazhagan Balasubramani; Venkatesh Anandpadmanabhan - Web page: https://patentgrep.com/patent/US12182024B2 ## CPC - G06F 12/0862: with prefetch - G06F 12/0868: Data transfer between cache memory and other subsystems, e.g. storage devices or host systems - G06F 12/08: in hierarchically structured memory systems, e.g. virtual memory systems - G06F 12/0882: Page mode - G06F 12/0897: with two or more cache hierarchy levels (with multilevel cache hierarchies G06F12/0811) - G06F 2212/1024: Latency reduction - G06F 2212/214: Solid state disk - G06F 2212/261: Storage comprising a plurality of storage devices - G06F 2212/311: In host system - G06F 2212/507: using speculative control - G06F 2212/6022: Using a prefetch buffer or dedicated prefetch cache - G06F 2212/6028: Prefetching based on hints or prefetch instructions - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system and method for a memory sub-system to reduce latency by prefetching data blocks and preloading them into host memory of a host system. An example system including a memory device and a processing device, operatively coupled with the memory device, to perform operations including: receiving a request of a host system to access a data block in the memory device; determining the data block stored in a first buffer in host memory is related to a set of one or more data blocks stored at the memory device; and storing the set of one or more data blocks in a second buffer in the host memory, wherein the first buffer is controlled by the host system and the second buffer is controlled by a memory sub-system. ## Claims ### Claim 1 A system comprising: a memory device; and a processing device operatively coupled to the memory device, to perform operations comprising: receiving a request of a host system to access a data block in the memory device; determining that the data block stored in a first buffer in a host memory is related to a set of one or more data blocks stored at the memory device; and storing the set of one or more data blocks in a second buffer in the host memory, wherein the first buffer is controlled by the host system and the second buffer is controlled by a memory sub-system. ### Claim 2 (depends on 1) The system of claim 1, wherein the processing device is a memory controller of the memory sub-system and wherein the first buffer comprises a page cache that is managed by the host system, and wherein the second buffer in the host memory comprises a Host Memory Buffer (HMB) that is exclusively controlled by the memory controller. ### Claim 3 (depends on 1) The system of claim 1, wherein the operations further comprise establishing the second buffer in the host memory, wherein the establishing comprises: transmitting, by the memory sub-system to the host system, an indication of a size of a region in the host memory; receiving, by the memory sub-system from the host system, a location of the region in the host memory; and updating the region to comprise the second buffer to store data blocks and to comprise a data structure indicating the data blocks stored in the second buffer. ### Claim 4 (depends on 1) The system of claim 1, wherein the operations further comprise transmitting the data block to the host memory responsive to receiving the request to access the data block, and transmitting the set of one or more data blocks to the host memory without receiving a request from the host system for any of the one or more data blocks. ### Claim 5 (depends on 1) The system of claim 1, wherein the processing device is included in a first level of a storage hierarchy that comprises secondary storage and prefetches data of the set and pushes the data of the set to a second level of the storage hierarchy that comprises the host memory as primary storage. ### Claim 6 (depends on 1) The system of claim 1, wherein the memory sub-system comprises a Solid State Drive (SSD) that comprises the processing device, RAM, and NAND, and wherein the processing device of the SSD prefetches a quantity of data from the NAND that exceeds a capacity of the RAM and stores the prefetched data in the second buffer in the host memory. ### Claim 7 (depends on 1) The system of claim 1, wherein the operations further comprise: receiving a plurality of write requests for the data block and at least one data block of the set, wherein each of the plurality of write requests comprise a particular stream identifier; and updating a data structure stored by the memory sub-system to indicate the data block and the at least one data block of the set are related to the particular stream identifier. ### Claim 8 (depends on 1) The system of claim 1, wherein determining the data block is related to the set of one or more data blocks comprises: accessing a data structure in the memory sub-system that comprises mapping data corresponding to location identifiers and stream identifiers, wherein the location identifiers comprise a Logical Block Address (LBA); and determining, based on the data structure, that the data block is related to each of the one or more data blocks in the set. ### Claim 9 (depends on 1) The system of claim 1, wherein the operations further comprise: transmitting, to the host system, a response indicating that the data block is stored in the first buffer in the host memory. ### Claim 10 A method comprising: receiving a request to access a data block in a memory device from a host system; determining that the data block stored in a first buffer in a host memory is related to a set of one or more data blocks stored at the memory device; and storing the set of one or more data blocks in a second buffer in the host memory, wherein the first buffer is controlled by the host system and the second buffer is controlled by a memory sub-system. ### Claim 11 (depends on 10) The method of claim 10, wherein the method is performed by a processing device that is a memory controller of the memory sub-system and wherein the first buffer comprises a page cache that is managed by the host system, and wherein the second buffer in the host memory comprises a Host Memory Buffer (HMB) that is exclusively controlled by the memory controller. ### Claim 12 (depends on 10) The method of claim 10, further comprising establishing the second buffer in the host memory, wherein the establishing comprises: transmitting, by the memory sub-system to the host system, an indication of a size of a region in host memory; receiving, by the memory sub-system from the host system, a location of the region in the host memory; and updating the region to comprise the second buffer to store data blocks and to comprise a data structure indicating the data blocks stored in the second buffer. ### Claim 13 (depends on 10) The method of claim 10, further comprising: transmitting the data block to the host memory responsive to receiving the request to access the data block; and transmitting the set of one or more data blocks to the host memory without receiving a request from the host system for any of the one or more data blocks. ### Claim 14 (depends on 10) The method of claim 10, wherein the memory sub-system comprises a Solid State Drive (SSD) that comprises the processing device, RAM, and NAND, and wherein the processing device of the SSD prefetches a quantity of data from the NAND that exceeds a capacity of the RAM and stores the prefetched data in the second buffer in the host memory. ### Claim 15 (depends on 10) The method of claim 10, further comprising: receiving a plurality of write requests for the data block and at least one data block of the set, wherein each of the plurality of write requests comprise a particular stream identifier; and updating a data structure stored by the memory sub-system to indicate the data block and the at least one data block of the set are related to the particular stream identifier. ### Claim 16 (depends on 10) The method of claim 10, wherein determining the data block is related to the set of one or more data blocks comprises: accessing a data structure in the memory sub-system that comprises mapping data corresponding to location identifiers and stream identifiers, wherein the location identifiers comprise a Logical Block Address (LBA); and determining, based on the data structure, that the data block is related to each of the one or more data blocks in the set. ### Claim 17 A non-transitory computer readable medium including instructions, which when performed by a computer system comprising a memory device, and a processing device operatively coupled to the memory device, cause the computer system to perform operations comprising: receiving a request to access a data block in the memory device from a host system; determining that the data block stored in a first buffer in a host memory is related to a set of one or more data blocks stored at the memory device; and storing the set of one or more data blocks in a second buffer in the host memory, wherein the first buffer is controlled by the host system and the second buffer is controlled by a memory sub-system. ### Claim 18 (depends on 17) The non-transitory computer readable medium of claim 17, wherein the processing device is a memory controller of the memory sub-system and wherein the first buffer comprises a page cache that is managed by the host system, and wherein the second buffer in the host memory comprises a Host Memory Buffer (HMB) that is exclusively controlled by the memory controller. ### Claim 19 (depends on 17) The non-transitory computer readable medium of claim 17, wherein the operations further comprise transmitting the data block to the host memory responsive to receiving the request to access the data block, and transmitting the set of one or more data blocks to the host memory without receiving a request from the host system for any of the one or more data blocks. ### Claim 20 (depends on 17) The non-transitory computer readable medium of claim 17, wherein the processing device is included in a first level of a storage hierarchy that comprises secondary storage and prefetches data of the set and pushes the data of the set to a second level of the storage hierarchy that comprises the host memory as primary storage.