Machine learning based tiered memory systems and methods

A method for dynamically placing data in a hybrid memory structure comprising dynamic random access memory (DRAM) and a persistent memory (PM) tier, the method comprising:

• i) monitoring, by a processor, a memory footprint of the DRAM;
• ii) monitoring, by the processor, a first plurality of data pages in the DRAM and periodically checking a respective data temperature of each data page of the first plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
• iii) categorizing, by the processor, each data page of the first plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above a threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
• iv) if the memory footprint of the DRAM exceeds a predetermined threshold memory footprint, moving, by the processor, any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of a second plurality of data pages;
• v) monitoring, by the processor, the second plurality of data pages in the PM tier and periodically checking a respective data temperature of each data page of the second plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
• vi) categorizing, by the processor, each data page of the second plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above the threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
• vii) moving, by the processor, any data pages of the second plurality of data pages categorized as hot into the DRAM to become part of the first plurality of data pages;
• viii) utilizing, by the processor, a reinforcement learning (RL) model to update the threshold data temperature;
• and
• ix) repeating steps i)-viii) while the method is being performed on the hybrid memory structure,
• the moving of any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of the second plurality of data pages comprising calculating a respective lifetime of each data page moved into the PM tier, the respective lifetime being equal to a current time minus a time said data page was created,
• the RL model using the following to update the threshold data temperature:
• an average data temperature of p coldest pages in the DRAM;
• an average data temperature of p hottest pages in the PM tier;
• an average lifetime of pages moved from DRAM to the PM tier in a previous window;
• and an average lifetime of pages moved from DRAM to the PM tier in a current window,
• p being an integer,
• the previous window being the previous time steps i)-viii) were performed, and
• the current window being the current time steps i)-viii) are being performed.

A system for dynamically placing data in a hybrid memory structure comprising dynamic random access memory (DRAM) and a persistent memory (PM) tier, the system comprising:

• a processor;
• and
• a machine-readable medium in operable communication with the processor and having instructions stored thereon that, when executed by the processor, perform the following steps:
• i) monitoring a memory footprint of the DRAM;
• ii) monitoring a first plurality of data pages in the DRAM and periodically checking a respective data temperature of each data page of the first plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
• iii) categorizing each data page of the first plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above a threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
• iv) if the memory footprint of the DRAM exceeds a predetermined threshold memory footprint, moving any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of a second plurality of data pages;
• v) monitoring the second plurality of data pages in the PM tier and periodically checking a respective data temperature of each data page of the second plurality of data pages, the respective data temperature of each data page being a measure of how often said data page is accessed;
• vi) categorizing each data page of the second plurality of data pages based on its respective data temperature, where a data page is categorized as hot if its data temperature is above the threshold data temperature and categorized as cold if its data temperature is at or below the threshold data temperature;
• vii) moving any data pages of the second plurality of data pages categorized as hot into the DRAM to become part of the first plurality of data pages;
• viii) utilizing a reinforcement learning (RL) model to update the threshold data temperature;
• and
• ix) repeating steps i)-viii) while the system is operating,
• the moving of any data pages of the first plurality of data pages categorized as cold into the PM tier to become part of the second plurality of data pages comprising calculating a respective lifetime of each data page moved into the PM tier, the respective lifetime being equal to a current time minus a time said data page was created,
• the RL model using the following to update the threshold data temperature:
• an average data temperature of p coldest pages in the DRAM;
• an average data temperature of p hottest pages in the PM tier;
• an average lifetime of pages moved from DRAM to the PM tier in a previous window;
• and an average lifetime of pages moved from DRAM to the PM tier in a current window,
• p being an integer,
• the previous window being the previous time steps i)-viii) were performed,
• the current window being the current time steps i)-viii) are being performed, and
• the instructions when executed further performing the following steps:
• moving any newly-created data pages to the DRAM to be part of the first plurality of data pages;
• monitoring a utilization of the PM tier;
• and
• if the utilization of the PM tier exceeds a predetermined PM tier capacity, either moving any data pages of the second plurality of data pages categorized as cold into a swap space of the hybrid memory structure or engaging an out-of-memory killer of the system to delete any data pages of the second plurality of data pages categorized as cold.