# US 2024/0256446 A1 · Multi-mode tiered memory cache controller

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0256446 A1
- Document ID: US20240256446A1
- Status: Published application
- Kind: A1
- Application number: 18/160,172
- Filed: 2023-01-26
- Published: 2024-08-01
- Earliest claimed: 2023-01-26
- Applicants: VMware LLC
- Inventors: Andreas Georg Nowatzyk; Pratap Subrahmanyam; Isam Wadih Akkawi; Adarsh Seethanadi Nayak; Nishchay Dua
- Web page: https://patentgrep.com/patent/US20240256446A1

## CPC

- G06F 12/0802: Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F 12/0897: with two or more cache hierarchy levels (with multilevel cache hierarchies G06F12/0811)
- G06F 12/0811: with multilevel cache hierarchies
- G06F 12/0868: Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
- G06F 12/0871: Allocation or management of cache space
- G06F 12/0888: using selective caching, e.g. bypass
- G06F 2212/1016: Performance improvement
- G06F 2212/205: Hybrid memory, e.g. using both volatile and non-volatile memory
- G06F 2212/30: Providing cache or TLB in specific location of a processing system
- G06F 2212/601: Reconfiguration of cache memory
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Techniques for implementing a hardware-based cache controller in, e.g., a tiered memory computer system are provided. In one set of embodiments, the cache controller can flexibly operate in a number of different modes that aid the OS/hypervisor of the computer system in managing and optimizing its use of the system's memory tiers. In another set of embodiments, the cache controller can implement a hardware architecture that enables it to significantly reduce the probability of tag collisions, decouple cache capacity management from cache lookup and allocation, and handle multiple concurrent cache transactions.

## Claims

### Claim 1

A method performed by a hardware-based tiered memory cache controller (TMCC) of a tiered memory computer system, the method comprising:
receiving a physical memory address for processing, wherein the tiered memory computer system includes a plurality of memory tiers ordered from highest to lowest, each memory tier corresponding to a type of physical memory available to the tiered memory computer system, wherein higher memory tiers correspond to types of physical memory that are faster but more expensive than lower memory tiers, and wherein the TMCC comprises a cache that is separate from the plurality of memory tiers;
determining, from among a plurality of operating modes supported by the TMCC, an operating mode associated with the physical memory address, wherein the determining is based on associations between physical memory address ranges and the plurality of operating modes, and wherein each operating mode performs one or more functions pertaining to management or usage of the plurality of memory tiers; and
processing the physical memory address in accordance with the determined operating mode.

### Claim 2 (depends on 1)

The method of claim 1 wherein the plurality of operating modes includes one or more operating modes that facilitate migration of memory objects between the plurality of memory tiers.

### Claim 3 (depends on 1)

The method of claim 1 wherein the TMCC resides between a dynamic random-access memory (DRAM) memory tier and one or more memory tiers below the DRAM memory tier, and wherein the plurality of operating modes includes a standard cache mode that involves caching memory objects held in the one or more memory tiers.

### Claim 4 (depends on 1)

The method of claim 1 wherein the plurality of operating modes includes a difference cache mode that involves caching writes made to a memory object held in a memory tier that is shared by a plurality of consumers and is subject to a copy-on-write (COW) policy, wherein the writes are made by a first consumer in the plurality of consumers, and wherein the cached writes are only visible to the first consumer and invisible to other consumers in the plurality of consumers.

### Claim 5 (depends on 1)

The method of claim 1 wherein the associations between the physical memory address ranges and the plurality of operating modes are defined by an operating system (OS) or hypervisor of the tiered memory computer system.

### Claim 6 (depends on 1)

The method of claim 1 wherein the TMCC is implemented using a field-programmable gate array, wherein the cache of the TMCC comprises a lookup and address map (LUAM), a metadata table separate from the LUAM, and a cache memory, wherein data held by the LUAM and metadata table is stored in static random-access memory (SRAM) of the FPGA, and wherein data held by the cache memory is stored in dynamic random-access memory of the FPGA.

### Claim 7 (depends on 6)

The method of claim 6 wherein the TMCC is capable of processing multiple transactions pertaining to a single physical memory address concurrently, and wherein the TMCC implements a component separate from the LUAM, the metadata table, and the cache memory that:
queues the multiple transactions to avoid conflicts; and
maintains state associated with the multiple transactions.

### Claim 8

A hardware-based tiered memory cache controller (TMCC) of a tiered memory computer system, the TMCC being configured to:
receive a physical memory address for processing, wherein the tiered memory computer system includes a plurality of memory tiers ordered from highest to lowest, each memory tier corresponding to a type of physical memory available to the tiered memory computer system, wherein higher memory tiers correspond to types of physical memory that are faster but more expensive than lower memory tiers, and wherein the TMCC comprises a cache that is separate from the plurality of memory tiers;
determine, from among a plurality of operating modes supported by the TMCC, an operating mode associated with the physical memory address, wherein the determining is based on associations between physical memory address ranges and the plurality of operating modes, and wherein each operating mode performs one or more functions pertaining to management or usage of the plurality of memory tiers; and
process the physical memory address in accordance with the determined operating mode.

### Claim 9 (depends on 8)

The TMCC of claim 8 wherein the plurality of operating modes includes one or more operating modes that facilitate migration of memory objects between the plurality of memory tiers.

### Claim 10 (depends on 8)

The TMCC of claim 8 wherein the TMCC resides between a dynamic random-access memory (DRAM) memory tier and one or more memory tiers below the DRAM memory tier, and wherein the plurality of operating modes includes a standard cache mode that involves caching memory objects held in the one or more memory tiers.

### Claim 11 (depends on 8)

The TMCC of claim 8 wherein the plurality of operating modes includes a difference cache mode that involves caching writes made to a memory object held in a memory tier that is shared by a plurality of consumers and is subject to a copy-on-write (COW) policy, wherein the writes are made by a first consumer in the plurality of consumers, and wherein the cached writes are only visible to the first consumer and invisible to other consumers in the plurality of consumers.

### Claim 12 (depends on 8)

The TMCC of claim 8 wherein the associations between the physical memory address ranges and the plurality of operating modes are defined by an operating system (OS) or hypervisor of the tiered memory computer system.

### Claim 13 (depends on 8)

The TMCC of claim 8 wherein the TMCC is implemented using a field-programmable gate array, wherein the cache of the TMCC comprises a lookup and address map (LUAM), a metadata table separate from the LUAM, and a cache memory, wherein data held by the LUAM and metadata table is stored in static random-access memory (SRAM) of the FPGA, and wherein data held by the cache memory is stored in dynamic random-access memory of the FPGA.

### Claim 14 (depends on 13)

The TMCC of claim 13 wherein the TMCC is capable of processing multiple transactions pertaining to a single physical memory address concurrently, and wherein the TMCC implements a component separate from the LUAM, the metadata table, and the cache memory that:
queues the multiple transactions to avoid conflicts; and
maintains state associated with the multiple transactions.

### Claim 15

A tiered memory computer system comprising:
a plurality of memory tiers ordered from highest to lowest, wherein higher memory tiers correspond to types of physical memory that are faster but more expensive than lower memory tiers; and
a tiered memory cache controller (TMCC) comprising a cache that is separate from the plurality of memory tiers, the TMCC being configured to:
receive a physical memory address for processing, wherein the tiered memory computer system includes;
determine, from among a plurality of operating modes supported by the TMCC, an operating mode associated with the physical memory address, wherein the determining is based on associations between physical memory address ranges and the plurality of operating modes, and wherein each operating mode performs one or more functions pertaining to management or usage of the plurality of memory tiers; and
process the physical memory address in accordance with the determined operating mode.

### Claim 16 (depends on 15)

The tiered memory computer system of claim 15 wherein the plurality of operating modes includes one or more operating modes that facilitate migration of memory objects between the plurality of memory tiers.

### Claim 17 (depends on 15)

The tiered memory computer system of claim 15 wherein the TMCC resides between a dynamic random-access memory (DRAM) memory tier and one or more memory tiers below the DRAM memory tier, and wherein the plurality of operating modes includes a standard cache mode that involves caching memory objects held in the one or more memory tiers.

### Claim 18 (depends on 15)

The tiered memory computer system of claim 15 wherein the plurality of operating modes includes a difference cache mode that involves caching writes made to a memory object held in a memory tier that is shared by a plurality of consumers and is subject to a copy-on-write (COW) policy, wherein the writes are made by a first consumer in the plurality of consumers, and wherein the cached writes are only visible to the first consumer and invisible to other consumers in the plurality of consumers.

### Claim 19 (depends on 15)

The tiered memory computer system of claim 15 wherein the associations between the physical memory address ranges and the plurality of operating modes are defined by an operating system (OS) or hypervisor of the tiered memory computer system.

### Claim 20 (depends on 15)

The tiered memory computer system of claim 15 wherein the TMCC is implemented using a field-programmable gate array, wherein the cache of the TMCC comprises a lookup and address map (LUAM), a metadata table separate from the LUAM, and a cache memory, wherein data held by the LUAM and metadata table is stored in static random-access memory (SRAM) of the FPGA, and wherein data held by the cache memory is stored in dynamic random-access memory of the FPGA.

### Claim 21 (depends on 20)

The tiered memory computer system of claim 20 wherein the TMCC is capable of processing multiple transactions pertaining to a single physical memory address concurrently, and wherein the TMCC implements a component separate from the LUAM, the metadata table, and the cache memory that:
queues the multiple transactions to avoid conflicts; and
maintains state associated with the multiple transactions.
