# US 11,307,986 B1 · Machine learning based tiered memory systems and methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,307,986 B1
- Document ID: US11307986B1
- Status: Granted
- Kind: B1
- Application number: 17/344,449
- Filed: 2021-06-10
- Granted: 2022-04-19
- Earliest claimed: 2021-06-10
- Assignees: The Florida International University Board of Trustees
- Applicants: Adnan Maruf; Janki Bhimani; Ashikee Ghosh; Raju Rangaswami
- Inventors: Adnan Maruf; Janki Bhimani; Ashikee Ghosh; Raju Rangaswami
- Web page: https://patentgrep.com/patent/US11307986B1

## CPC

- G06F 12/0804: with main memory updating (G06F12/0806 takes precedence)
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06F 12/0868: Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
- G06N 20/00: Machine learning
- G06F 2212/1021: Hit rate improvement
- G06F 2212/1024: Latency reduction
- G06F 2212/1028: Power efficiency
- G06F 2212/214: Solid state disk
- G06F 2212/502: using adaptive policy
- G06F 2212/7205: Cleaning, compaction, garbage collection, erase control
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

Systems and methods for dynamically placing data in a hybrid memory structure are provided. A machine learning (ML)-based, adaptive tiered memory system can actively monitor application memory to dynamically place the right data in the right memory tier at the right time. The memory system can use reinforcement learning to perform dynamic tier placement of memory pages.

## Claims

### Claim 1

A system for dynamically placing data in a hybrid memory structure comprising dynamic random access memory (DRAM) and a persistent memory (PM) tier, the system comprising:
a processor; and
a machine-readable medium in operable communication with the processor and having instructions stored thereon that, when executed by the processor, perform the following steps:
i) monitoring a memory footprint of the DRAM;
ii) monitoring a first plurality of data pages in the DRAM and periodically checking a respective data temperature of each data page of the first plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
iii) categorizing each data page of the first plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above a threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
iv) if the memory footprint of the DRAM exceeds a predetermined threshold memory footprint, moving any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of a second plurality of data pages;
v) monitoring the second plurality of data pages in the PM tier and periodically checking a respective data temperature of each data page of the second plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
vi) categorizing each data page of the second plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above the threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
vii) moving any data pages of the second plurality of data pages categorized as hot into the DRAM to become part of the first plurality of data pages;
viii) utilizing a reinforcement learning (RL) model to update the threshold data temperature; and
ix) repeating steps i)-viii) while the system is operating,
the moving of any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of the second plurality of data pages comprising calculating a respective lifetime of each data page moved into the PM tier, the respective lifetime being equal to a current time minus a time said data page was created,
the RL model using the following to update the threshold data temperature: an average data temperature of p coldest pages in the DRAM; an average data temperature of p hottest pages in the PM tier; an average lifetime of pages moved from DRAM to the PM tier in a previous window; and an average lifetime of pages moved from DRAM to the PM tier in a current window,
p being an integer,
the previous window being the previous time steps i)-viii) were performed, and
the current window being the current time steps i)-viii) are being performed.

### Claim 2 (depends on 1)

The system according to claim 1, the instructions when executed further performing the following step:
moving any newly-created data pages to the DRAM to be part of the first plurality of data pages.

### Claim 3 (depends on 2)

The system according to claim 2, the instructions when executed further performing the following steps:
monitoring a utilization of the PM tier; and
if the utilization of the PM tier exceeds a predetermined PM tier capacity, moving any data pages of the second plurality of data pages categorized as cold into a swap space of the hybrid memory structure.

### Claim 4 (depends on 2)

The system according to claim 2, the instructions when executed further performing the following steps:
monitoring a utilization of the PM tier; and
if the utilization of the PM tier exceeds a predetermined PM tier capacity, engaging an out-of-memory killer of the system to delete any data pages of the second plurality of data pages categorized as cold.

### Claim 5 (depends on 1)

The system according to claim 1, the instructions when executed further performing the following step:
moving any newly-created data pages to the DRAM to be part of the first plurality of data pages.

### Claim 6 (depends on 1)

The system according to claim 1, the instructions when executed further performing the following steps:
monitoring a utilization of the PM tier; and
if the utilization of the PM tier exceeds a predetermined PM tier capacity, moving any data pages of the second plurality of data pages categorized as cold into a swap space of the hybrid memory structure.

### Claim 7 (depends on 1)

The system according to claim 1, the instructions when executed further performing the following steps:
monitoring a utilization of the PM tier; and
if the utilization of the PM tier exceeds a predetermined PM tier capacity, engaging an out-of-memory killer of the system to delete any data pages of the second plurality of data pages categorized as cold.

### Claim 8 (depends on 1)

The system according to claim 1, the RL model using an average data temperature of p coldest pages in the DRAM and an average data temperature of p hottest pages in the PM tier to update the threshold data temperature, p being an integer.

### Claim 9

A method for dynamically placing data in a hybrid memory structure comprising dynamic random access memory (DRAM) and a persistent memory (PM) tier, the method comprising:
i) monitoring, by a processor, a memory footprint of the DRAM;
ii) monitoring, by the processor, a first plurality of data pages in the DRAM and periodically checking a respective data temperature of each data page of the first plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
iii) categorizing, by the processor, each data page of the first plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above a threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
iv) if the memory footprint of the DRAM exceeds a predetermined threshold memory footprint, moving, by the processor, any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of a second plurality of data pages;
v) monitoring, by the processor, the second plurality of data pages in the PM tier and periodically checking a respective data temperature of each data page of the second plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
vi) categorizing, by the processor, each data page of the second plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above the threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
vii) moving, by the processor, any data pages of the second plurality of data pages categorized as hot into the DRAM to become part of the first plurality of data pages;
viii) utilizing, by the processor, a reinforcement learning (RL) model to update the threshold data temperature; and
ix) repeating steps i)-viii) while the method is being performed on the hybrid memory structure,
the moving of any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of the second plurality of data pages comprising calculating a respective lifetime of each data page moved into the PM tier, the respective lifetime being equal to a current time minus a time said data page was created,
the RL model using the following to update the threshold data temperature: an average data temperature of p coldest pages in the DRAM; an average data temperature of p hottest pages in the PM tier; an average lifetime of pages moved from DRAM to the PM tier in a previous window; and an average lifetime of pages moved from DRAM to the PM tier in a current window,
p being an integer,
the previous window being the previous time steps i)-viii) were performed, and
the current window being the current time steps i)-viii) are being performed.

### Claim 10 (depends on 9)

The method according to claim 9, further comprising moving, by the processor, any newly-created data pages to the DRAM to be part of the first plurality of data pages.

### Claim 11 (depends on 10)

The method according to claim 10, further comprising:
monitoring, by the processor, a utilization of the PM tier; and
if the utilization of the PM tier exceeds a predetermined PM tier capacity, moving, by the processor, any data pages of the second plurality of data pages categorized as cold into a swap space of the hybrid memory structure.

### Claim 12 (depends on 10)

The method according to claim 10, the instructions when executed further performing the following steps:
monitoring, by the processor, a utilization of the PM tier; and
if the utilization of the PM tier exceeds a predetermined PM tier capacity, engaging, by the processor, an out-of-memory killer of the system to delete any data pages of the second plurality of data pages categorized as cold.

### Claim 13 (depends on 9)

The method according to claim 9, further comprising moving, by the processor, any newly-created data pages to the DRAM to be part of the first plurality of data pages.

### Claim 14 (depends on 9)

The method according to claim 9, further comprising:
monitoring, by the processor, a utilization of the PM tier; and
if the utilization of the PM tier exceeds a predetermined PM tier capacity, moving, by the processor, any data pages of the second plurality of data pages categorized as cold into a swap space of the hybrid memory structure.

### Claim 15 (depends on 9)

The method according to claim 9, further comprising:
monitoring, by the processor, a utilization of the PM tier; and
if the utilization of the PM tier exceeds a predetermined PM tier capacity, engaging, by the processor, an out-of-memory killer of the system to delete any data pages of the second plurality of data pages categorized as cold.

### Claim 16

A system for dynamically placing data in a hybrid memory structure comprising dynamic random access memory (DRAM) and a persistent memory (PM) tier, the system comprising:
a processor; and
a machine-readable medium in operable communication with the processor and having instructions stored thereon that, when executed by the processor, perform the following steps:
i) monitoring a memory footprint of the DRAM;
ii) monitoring a first plurality of data pages in the DRAM and periodically checking a respective data temperature of each data page of the first plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
iii) categorizing each data page of the first plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above a threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
iv) if the memory footprint of the DRAM exceeds a predetermined threshold memory footprint, moving any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of a second plurality of data pages;
v) monitoring the second plurality of data pages in the PM tier and periodically checking a respective data temperature of each data page of the second plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
vi) categorizing each data page of the second plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above the threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
vii) moving any data pages of the second plurality of data pages categorized as hot into the DRAM to become part of the first plurality of data pages;
viii) utilizing a reinforcement learning (RL) model to update the threshold data temperature; and
ix) repeating steps i)-viii) while the system is operating,
the moving of any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of the second plurality of data pages comprising calculating a respective lifetime of each data page moved into the PM tier, the respective lifetime being equal to a current time minus a time said data page was created,
the RL model using the following to update the threshold data temperature: an average data temperature of p coldest pages in the DRAM; an average data temperature of p hottest pages in the PM tier; an average lifetime of pages moved from DRAM to the PM tier in a previous window; and an average lifetime of pages moved from DRAM to the PM tier in a current window,
p being an integer,
the previous window being the previous time steps i)-viii) were performed,
the current window being the current time steps i)-viii) are being performed, and
the instructions when executed further performing the following steps:
moving any newly-created data pages to the DRAM to be part of the first plurality of data pages;
monitoring a utilization of the PM tier; and
if the utilization of the PM tier exceeds a predetermined PM tier capacity, either moving any data pages of the second plurality of data pages categorized as cold into a swap space of the hybrid memory structure or engaging an out-of-memory killer of the system to delete any data pages of the second plurality of data pages categorized as cold.