Prefetching and caching between high bandwidth memory (hbm) and non-volatile (nvm) cache on processing die

A method comprising:

• separating read commands, received from a compute die of a compute device, according to a plurality of processing threads executed by the compute die;
• predicting, by a first prediction engine for a first processing thread, of the plurality of processing threads, a first data access pattern associated with a first read command received from the first processing thread, wherein the first data access pattern is in relation to a on-chip cache comprising one or more volatile memory dies and one or more non-volatile memory dies;
• determining, by a first performance monitor associated with the first prediction engine, a first statistical value corresponding to a likelihood that the first prediction engine successfully predicted the first data access pattern;
• and
• responsive to the first statistical value satisfying a threshold value, forwarding, by the first performance monitor, a prefetch command consistent with the first data access pattern to a prefetch command manager for handling.

A non-transitory computer-readable storage medium comprising executable instructions that, when executed by a memory controller of a compute device with a on-chip cache comprising one or more volatile memory dies and one or more non-volatile memory dies, cause the memory controller to perform operations comprising:

• separating read commands, received from a compute die of a compute device, according to a plurality of processing threads executed by the compute die;
• predicting, by a first prediction engine for a first processing thread, of the plurality of processing threads, a first data access pattern associated with a first read command received from the first processing thread, wherein the first data access pattern is in relation to a on-chip cache comprising one or more volatile memory dies and one or more non-volatile memory dies;
• determining, by a first performance monitor associated with the first prediction engine, a first statistical value corresponding to a likelihood that the first prediction engine successfully predicted the first data access pattern;
• and
• responsive to the first statistical value satisfying a threshold value, forwarding, by the first performance monitor, a prefetch command consistent with the first data access pattern to a prefetch command manager for handling.