Caching techniques for migrating and replicating data

A system comprising:

• a processor;
• and
• a memory comprising code stored thereon that, when executed, performs a method of processing data comprising:
• receiving a hierarchical structure of metadata (MD) pages for a logical device;
• and
• performing first processing to copy data stored on the logical device from a source system to a target system, the first processing including:
• determining a sequence of the MD pages of the logical device in accordance with a depth first traversal of the hierarchical structure;
• defining a cache management policy in accordance with the sequence that indicates, at various points in time, when to load the MD pages into a cache and when to remove the MD pages from the cache;
• loading the MD pages into the cache and removing MD pages from the cache at the various points in time and in accordance with the cache management policy;
• and
• copying data pages stored at logical addresses of the logical device from the source system to the target system, wherein said copying of the data pages is performed in an order in which the logical addresses are accessed using one or more of the MD pages stored in the cache at the various points in time in accordance with the cache management policy, wherein the cache management policy includes one or more rules that indicate to prefetch one or more MD pages of the hierarchical structure into the cache in accordance with a sequential order in which the MD pages of the sequence are accessed, and wherein the cache management policy further indicates that M, a maximum number of MD pages allowed to be stored in the cache, is equal to a mathematical sum of N and P, wherein N denotes a maximum number of levels in the hierarchical structure, and wherein P denotes a maximum number of prefetched MD pages allowed to be stored in the cache, wherein the cache management policy indicates that each parent MD page of the hierarchical structure remains resident in the cache until processing of all child MD pages of said each parent MD page of the hierarchical structure has completed, and wherein said copying the data pages further comprises:
• responsive to loading a first leaf MD page of the sequence into the cache, copying a first plurality of the data pages referenced by the first leaf MD page from the source system to the target system.

A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of processing data comprising:

• receiving a hierarchical structure of metadata (MD) pages for a logical device;
• and
• performing first processing to copy data stored on the logical device from a source system to a target system, the first processing including:
• determining a sequence of the MD pages of the logical device in accordance with a depth first traversal of the hierarchical structure;
• defining a cache management policy in accordance with the sequence that indicates, at various points in time, when to load the MD pages into a cache and when to remove the MD pages from the cache;
• loading the MD pages into the cache and removing MD pages from the cache at the various points in time and in accordance with the cache management policy;
• and
• copying data pages stored at logical addresses of the logical device from the source system to the target system, wherein said copying of the data pages is performed in an order in which the logical addresses are accessed using one or more of the MD pages stored in the cache at the various points in time in accordance with the cache management policy, wherein the cache management policy includes one or more rules that indicate to prefetch one or more MD pages of the hierarchical structure into the cache in accordance with a sequential order in which the MD pages of the sequence are accessed, and wherein the cache management policy further indicates that M, a maximum number of MD pages allowed to be stored in the cache, is equal to a mathematical sum of N and P, wherein N denotes a maximum number of levels in the hierarchical structure, and wherein P denotes a maximum number of prefetched MD pages allowed to be stored in the cache, wherein the cache management policy indicates that each parent MD page of the hierarchical structure remains resident in the cache until processing of all child MD pages of said each parent MD page of the hierarchical structure has completed, and wherein said copying the data pages further comprises:
• responsive to loading a first leaf MD page of the sequence into the cache, copying a first plurality of the data pages referenced by the first leaf MD page from the source system to the target system.

A method of processing data comprising:

• receiving a hierarchical structure of metadata (MD) pages for a logical device;
• and
• performing first processing to copy data stored on the logical device from a source system to a target system, the first processing including:
• determining a sequence of the MD pages of the logical device in accordance with a depth first traversal of the hierarchical structure;
• defining a cache management policy in accordance with the sequence that indicates, at various points in time, when to load the MD pages into a cache of the source system and when to remove the MD pages from the cache of the source system;
• loading the MD pages into the cache of the source system and removing MD pages from the cache of the source system at the various points in time and in accordance with the cache management policy;
• and
• copying data pages stored at logical addresses of the logical device from the source system to the target system, wherein said copying of the data pages is performed in an order in which the logical addresses are accessed using one or more of the MD pages stored in the cache at the various points in time in accordance with the cache management policy, wherein the MD pages of the hierarchical structure include leaf MD pages and the leaf MD pages access the data pages stored on the logical device, and wherein the data pages accessed by the leaf MD pages are directly copied, over a network connection, from one or more physical storage devices of the source system to one or more other physical storage devices of the target system without storing the data pages in the cache of the source system.

A system comprising:

• one or more processors;
• and
• a memory comprising code stored therein that, when executed, performs a method of processing data comprising:
• receiving a hierarchical structure of metadata (MD) pages for a logical device;
• and
• performing first processing to copy data stored on the logical device from a source system to a target system, the first processing including:
• determining a sequence of the MD pages of the logical device in accordance with a depth first traversal of the hierarchical structure;
• defining a cache management policy in accordance with the sequence that indicates, at various points in time, when to load the MD pages into a cache of the source system and when to remove the MD pages from the cache of the source system;
• loading the MD pages into the cache of the source system and removing MD pages from the cache of the source system at the various points in time and in accordance with the cache management policy;
• and
• copying data pages stored at logical addresses of the logical device from the source system to the target system, wherein said copying of the data pages is performed in an order in which the logical addresses are accessed using one or more of the MD pages stored in the cache at the various points in time in accordance with the cache management policy, wherein the MD pages of the hierarchical structure include leaf MD pages and the leaf MD pages access the data pages stored on the logical device, and wherein the data pages accessed by the leaf MD pages are directly copied, over a network connection, from one or more physical storage devices of the source system to one or more other physical storage devices of the target system without storing the data pages in the cache of the source system.

A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of processing data comprising:

• receiving a hierarchical structure of metadata (MD) pages for a logical device;
• and
• performing first processing to copy data stored on the logical device from a source system to a target system, the first processing including:
• determining a sequence of the MD pages of the logical device in accordance with a depth first traversal of the hierarchical structure;
• defining a cache management policy in accordance with the sequence that indicates, at various points in time, when to load the MD pages into a cache of the source system and when to remove the MD pages from the cache of the source system;
• loading the MD pages into the cache of the source system and removing MD pages from the cache of the source system at the various points in time and in accordance with the cache management policy;
• and
• copying data pages stored at logical addresses of the logical device from the source system to the target system, wherein said copying of the data pages is performed in an order in which the logical addresses are accessed using one or more of the MD pages stored in the cache at the various points in time in accordance with the cache management policy, wherein the MD pages of the hierarchical structure include leaf MD pages and the leaf MD pages access the data pages stored on the logical device, and wherein the data pages accessed by the leaf MD pages are directly copied, over a network connection, from one or more physical storage devices of the source system to one or more other physical storage devices of the target system without storing the data pages in the cache of the source system.