Partitioning mechanism for parallel processing in delta generation

A non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• receiving, by a computing system, a request to perform a first cross-region replication between a source file system in a source region and a target file system in a target region, the first cross-region replication being one of a plurality of parallel replications comprising i) a first-level operation comprising extracting deltas representing differences between a first snapshot and a second snapshot of the source file system and ii) a plurality of second-level operations comprising partitioning, in parallel, respective key-value pairs into key ranges, and a first result of the first-level operation corresponding to a second result of the plurality of second-level operations;
• performing, by the computing system, the first-level operation by extracting the deltas representing the differences between the first snapshot and the second snapshot of the source file system, the deltas being stored in a tree structure associated with the source file system in the source region, the tree structure having multiple levels of hierarchy comprising more nodes in a lower hierarchal level than a higher hierarchal level, and the first snapshot and the second snapshot comprising the respective sets of key-value pairs;
• partitioning, by the computing system, the respective sets of key-value pairs identified in the first-level operation into a first number of key partitions by traversing the tree structure from a top level of the tree structure downward, the first number being configured to be adjusted for each replication cycle;
• and
• performing, by the computing system, the plurality of second-level operations on the first number of key partitions in parallel, each second-level operation of the plurality of second-level operations being performed on a key partition of the first number of key partitions.

A computing system, comprising:

• one or more processors;
• and
• one or more computer readable media storing computer-executable instructions that, when executed by the one or more processors, cause the system to:
• receive a request to perform a first cross-region replication between a source file system in a source region and a target file system in a target region, the first cross-region replication being one of a plurality of parallel replications comprising i) a first-level operation comprising extracting deltas representing differences between a first snapshot and a second snapshot of the source file system and ii) a plurality of second-level operations comprising partitioning, in parallel, respective key-value pairs into key ranges, and a first result of the first-level operation corresponding to a second result of the plurality of second-level operations;
• perform the first-level operation by extracting the deltas representing the differences between the first snapshot and the second snapshot of the source file system, the deltas being stored in a tree structure associated with the source file system in the source region, the tree structure having multiple levels of hierarchy comprising more nodes in a lower hierarchal level than a higher hierarchal level, and the first snapshot and the second snapshot comprising the respective sets of key-value pairs;
• partition the respective sets of key-value pairs identified in the first-level operation into a first number of key partitions by traversing the tree structure from a top level of the tree structure downward, the first number being configured to be adjusted for each replication cycle;
• and
• perform the plurality of second-level operations on the first number of key partitions in parallel, each second-level operation of the plurality of second-level operations being performed on a key partition of the first number of key partitions.