Methods and apparatus for managing data flow in a continuous data replication system having journaling

An article, comprising:

• a storage medium having stored thereon instructions that when executed by a machine result in the following:
• journaling replication data using a number of journal processing stages;
• receiving a data transfer request from a first component at a transmitter module in a continuous data replication system having a production site and a backup site, the production site having the transmitter module and a transmitter credit mechanism, the transmitter module is configured to transmit data over a network for replication in the backup site;
• modifying a flow of credits from the transmitter credit mechanism to the first component to reduce or stop data flow to the transmitter module if a high-load condition is detected at the receiver module;
• modifying a flow of credits from a receiver credit mechanism to the transmitter module to reduce or stop data flow to the receiver module if a high-load condition is detected at a receiver module;
• modifying a flow of credits from a distributor credit mechanism to the receiver module to reduce or stop data flow to the distributor module if a high-load condition is detected at a receiver module;
• and
• reducing the number of journal processing stages for replicating data if a high-load condition is detected before activating the transmitter credit mechanism.

A data replication system, comprising:

• a first data site having a transmitter module to receive data transactions;
• a backup data site having a receiver module to receive the data transactions to be replicated and a distributor module to perform journaling of data from the receiver module, the system including a memory to store instructions to enable:
• (a) receiving write data transactions from the first data site to be backed up at the backup data site;
• (b) writing the write data to a first position in a do data stream;
• (c) writing metadata for the write data to a first position in a do metadata stream, wherein the metadata for the write data in step (c) comprises a first field to store a pointer to an offset in the do data stream where the write data is located;
• (d) reading data from a second position previously stored in the do data stream;
• (e) reading metadata from a second position previously stored in the do metadata stream;
• (f) reading data to be overwritten from a logical unit in the backup storage site, where location and size of the data to be overwritten is determined from the do metadata stream;
• (g) writing the data to be overwritten to a first position in an undo data stream;
• (h) writing metadata for the data to be overwritten to an undo metadata stream wherein the metadata for the data to be overwritten comprises a first field to store a pointer to an offset in the undo data stream where the data to be overwritten is located;
• (i) writing the write data to the logical unit in the backup storage site;
• and
• (j) managing a flow of credits between the transmitter module, the receiver module, and the distributor module to manage data flow in one or more of steps (a)-(i).