# US 12,001,404 B2 · Techniques for replication checkpointing during disaster recovery

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,001,404 B2
- Document ID: US12001404B2
- Status: Granted
- Kind: B2
- Application number: 18/304,161
- Filed: 2023-04-20
- Published: 2023-12-21
- Granted: 2024-06-04
- Earliest claimed: 2022-06-16
- Assignees: Oracle International Corporation
- Applicants: Oracle International Corporation
- Inventors: Satish Kumar Kashi Visvanathan; Viggnesh Venugopal; Victor Vladimir Golosovker; Ravi Lingappa Shamanna
- Web page: https://patentgrep.com/patent/US12001404B2

## CPC

- G06F 16/178: Techniques for file synchronisation in file systems
- G06F 9/505: {considering the load}
- G06F 11/1417: {Boot up procedures}
- G06F 11/1451: {by selection of backup contents}
- G06F 11/1464: {for networked environments}
- G06F 11/2023: {Failover techniques}
- G06F 11/2028: {eliminating a faulty processor or activating a spare}
- G06F 11/2038: {with a single idle spare processing component}
- G06F 11/2094: {Redundant storage or storage space (G06F11/2056 takes precedence)}
- G06F 11/2097: {maintaining the standby controller/processing unit updated (initialisation or re-synchronisation thereof G06F11/1658 and subgroups)}
- G06F 16/11: File system administration, e.g. details of archiving or snapshots (error detection or correction of the data by redundancy in operations G06F11/14)
- G06F 16/119: {Details of migration of file systems (migration mechanisms in storage systems G06F3/0647)}
- G06F 16/128: {Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion (error detection or correction of the data by redundancy in operations or in hardware G06F11/14, G06F11/16)}
- G06F 16/1756: {based on delta files}
- G06F 16/1844: {Management specifically adapted to replicated file systems}
- G06F 16/185: Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof (details of archiving G06F16/11)
- G06F 16/2365: {Ensuring data consistency and integrity}
- G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F 21/602: {Providing cryptographic facilities or services}
- G06F 21/6218: {to a system of files or objects, e.g. local or distributed file system or database}
- H04L 9/0819: {Key transport or distribution, i.e. key establishment techniques where one party creates or otherwise obtains a secret value, and securely transfers it to the other(s) (network architectures or network communication protocols for key distribution in a packet data network H04L63/062)}
- H04L 9/0836: {using tree structure or hierarchical structure}
- H04L 9/0891: {Revocation or update of secret information, e.g. encryption key update or rekeying}
- H04L 9/0894: {Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage}
- H04L 9/14: using a plurality of keys or algorithms
- H04L 9/3228: {One-time or temporary data, i.e. information which is sent for every authentication or authorization, e.g. one-time-password, one-time-token or one-time-key}
- G06F 11/2048: {where the redundant components share neither address space nor persistent storage}
- G06F 16/1774: {Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files}
- G06F 16/2246: {Trees, e.g. B+trees}
- G06F 2201/84: Using snapshots, i.e. a logical point-in-time copy of the data
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Techniques are described for checkpointing multiple key ranges in parallel and concurrently during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, multiple range threads processing multiple key ranges, one thread per key range, create checkpoints for their respective key ranges in parallel and concurrently after processing a per-determined number of B-tree keys. In certain embodiments, each thread requests a lock from a central checkpoint record and takes turns for updating a status byte while continuing processing the B-tree keys in its responsible key range. In certain embodiments, upon encountering a failure event, either a system crash or a thread failure, each thread restarts its B-tree key processing from a B-tree key after the most recent checkpoint.

## Claims

### Claim 1

A method, comprising:
receiving, by a computing system, a request for a file system replication between a source file system and a target file system, the source file system and the target file system being in different regions;
creating, by a first processing thread of the computing system, a first checkpoint in a first key range after fulfilling a predetermined requirement for processing first binary tree (B-tree) key-value pairs in the first key range;
requesting, by the first processing thread of the computing system, a lock to update a central record after creating the first checkpoint in the first key range;
creating, by a second processing thread of the computing system, a second checkpoint in a second key range after fulfilling the predetermined requirement for processing second B-tree key-value pairs in the second key range, wherein the processing the first B-tree key-value pairs in the first key range by the first processing thread is performed in parallel with the processing the second B-tree key-value pairs in the second key range by the second processing thread;
requesting, by the second processing thread of the computing system, the lock to update the central record after creating the second checkpoint in the second key range; and
granting, by the computing system, the lock to the first processing thread to update the central record, the request by the first processing thread being earlier than the request by the second processing thread, the lock being configured to prevent the second process thread from updating the central record.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
processing, by the first processing thread, more B-tree key-value pairs in the first key range after updating the central record; and
granting, by the computing system, the lock to the second processing thread to update the central record after the first processing thread completes updating the central record.

### Claim 3 (depends on 1)

The method of claim 1, further comprising processing, by the second processing thread, more B-tree key-value pairs in the second key range while waiting for the lock.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first processing thread operates independently from the second processing thread.

### Claim 5 (depends on 1)

The method of claim 1, further comprising restarting processing the first B-tree key-value pairs in the first key range by the first processing thread from a B-tree key after the first checkpoint in the first key range upon detecting a system failure.

### Claim 6 (depends on 1)

The method of claim 1, further comprising restarting processing the first B-tree key-value pairs in the first key range by a third processing thread from a B-tree key after the first checkpoint in the first key range upon detecting a failure of the first processing thread.

### Claim 7

A non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors of a computing system, cause the one or more processors to perform operations comprising:
receiving, by the computing system, a request for a file system replication between a source file system and a target file system, the source file system and the target file system being in different regions;
creating, by a first processing thread of the computing system, a first checkpoint in a first key range after fulfilling a predetermined requirement for processing first binary tree (B-tree) key-value pairs in the first key range;
requesting, by the first processing thread of the computing system, a lock to update a central record after creating the first checkpoint in the first key range;
creating, by a second processing thread of the computing system, a second checkpoint in a second key range after fulfilling a predetermined requirement for processing second B-tree key-value pairs in the second key range, wherein the processing the first B-tree key-value pairs in the first key range by the first processing thread is performed in parallel with the processing the second B-tree key-value pairs in the second key range by the second processing thread;
requesting, by the second processing thread of the computing system, the lock to update the central record after creating the second checkpoint in the second key range; and
granting, by the computing system, the lock to the first processing thread to update the central record, the request by the first processing thread being earlier than the request by the second processing thread, the lock being configured to prevent the second process thread from updating the central record.

### Claim 8 (depends on 7)

The non-transitory computer-readable medium of claim 7, the operations further comprising:
processing, by the first processing thread, more B-tree key-value pairs in the first key range after updating the central record; and
granting, by the computing system, the lock to the second processing thread to update the central record after the first processing thread completes updating the central record.

### Claim 9 (depends on 7)

The non-transitory computer-readable medium of claim 7, wherein the first processing thread operates independently from the second processing thread.

### Claim 10 (depends on 7)

The non-transitory computer-readable medium of claim 7, the operations further comprising restarting processing the first B-tree key-value pairs in the first key range by the first processing thread from a B-tree key after the first checkpoint in the first key range upon detecting a system failure.

### Claim 11 (depends on 7)

The non-transitory computer-readable medium of claim 7, the operations further comprising restarting processing the first B-tree key-value pairs in the first key range by a third processing thread from a B-tree key after the first checkpoint in the first key range upon detecting a failure of the first processing thread.

### Claim 12

A system, comprising:
one or more processors; and
one or more computer readable media storing computer-executable instructions that, when executed by the one or more processors, cause the system to:
receive a request for a file system replication between a source file system and a target file system, the source file system and the target file system being in different regions;
create, by a first processing thread of the system, a first checkpoint in a first key range after fulfilling a predetermined requirement for processing first binary tree (B-tree) key-value pairs in the first key range;
request, by the first processing thread of the system, a lock to update a central record after creating the first checkpoint in the first key range;
create, by a second processing thread of the system, a second checkpoint in a second key range after fulfilling a predetermined requirement for processing second B-tree key-value pairs in the second key range, wherein the processing the first B-tree key-value pairs in the first key range by the first processing thread is performed in parallel with the processing the second B-tree key-value pairs in the second key range by the second processing thread;
request, by the second processing thread, the lock to update the central record after creating the second checkpoint in the second key range; and
grant the lock to the first processing thread to update the central record, the request by the first processing thread being earlier than the request by the second processing thread, the lock being configured to prevent the second process thread from updating the central record.

### Claim 13 (depends on 12)

The system of claim 12, wherein the system is further caused to:
process, by the first processing thread of the system, more B-tree key-value pairs in the first key range after updating the central record; and
grant the lock to the second processing thread to update the central record after the first processing thread completes updating the central record.

### Claim 14 (depends on 12)

The system of claim 12, wherein the first processing thread operates independently from the second processing thread.

### Claim 15 (depends on 12)

The system of claim 12, wherein the system is further caused to restart processing the first B-tree key-value pairs in the first key range by the first processing thread from a B-tree key after the first checkpoint in the first key range upon detecting a system failure.

### Claim 16 (depends on 12)

The system of claim 12, wherein the system is further caused to restart processing the first B-tree key-value pairs in the first key range by a third processing thread from a B-tree key after the first checkpoint in the first key range upon detecting a failure of the first processing thread.