Data replication in site recovery environment

A non-transitory computer-readable medium containing computer program code that, when executed, performs an operation for updating a second database included in a second database management system of a site recovery platform to be consistent with a first database included in a first database management system of the site recovery platform by sending changes from the first database to the second database, the operation comprising:

• using a first thread of a plurality of threads:
• detecting a write operation modifying one or more property values of one or more properties of one or more data objects in the first database, the first database being a key-value database;
• and
• while at least one of the one or more data objects within the first database is locked as a result of the detected write operation, writing first object change data;
• based on the detected write operation, to a journal table of the first database, wherein the writing the first object change data creates a record within the journal table of the first database using an append operation, such that no lock on the journal table of the first database is required to prevent a resource conflict between the first thread and other threads in the plurality of threads, and wherein the record written into the journal table includes an identification of the one or more data objects and not the one or more property values of the properties of the one or more data objects themselves;
• using a second one or more threads distinct from the plurality of threads:
• querying the journal table of the first database to retrieve a portion of all object change data in the journal table, the portion corresponding to a first window of time, the portion comprising a plurality of entries including unneeded entries, the unneeded entries including any duplicate entries and any entries no longer indicative of a current state of any data object in the first database;
• processing the retrieved portion to create processed object change data by removing the unneeded entries from the plurality of entries;
• retrieving object data from the first database, the object data including a particular set of property values in the first database of any changed data objects identified in the processed object change data;
• and
• sending the retrieved object data to the second database management system, the second database being a database with a single table;
• and
• receiving the retrieved object data, processing the particular set of property values in the retrieved object data, and writing all of the property values in the particular set of property values to the single table of the second database, whereby the second database is updated to reflect changes in the first database.

A system comprising:

• a processor;
• and
• a memory containing a program that, when executed by the processor, performs an operation for updating a second database included in a second database management system of a site recovery platform to be consistent with a first database included in a first database management system of a site recovery platform by sending changes from the first database to the second database, the operation comprising:
• using a first thread of a plurality of threads:
• detecting a write operation modifying one or more property values of one or more properties of one or more data objects in the first database, the first database being a key-value database;
• and
• while at least one of the one or more data objects within the first database is locked as a result of the detected write operation, writing first object change data, based on the detected write operation, to a journal table of the first database, wherein the writing the first object change data creates a record within the journal table of the first database using an append operation, such that no lock on the journal table of the first database is required to prevent a resource conflict between the first thread and other threads in the plurality of threads, and wherein the record written into the journal table includes an identification of the one or more data objects and not the one or more property values of the properties of the one or more data objects themselves;
• using a second one or more threads distinct from the plurality of threads:
• querying the journal table of the first database system to retrieve a portion of all object change data in the journal table, the portion corresponding to a first window of time, the portion comprising a plurality of entries including unneeded entries, the unneeded entries including any duplicate entries and any entries no longer indicative of a current state of any data object in the first database;
• processing the retrieved portion to create processed object change data by removing the unneeded entries from the plurality of entries;
• retrieving object data from the first database, the object data including a particular set of property values in the first database of any changed data objects identified in the processed object change data;
• sending the retrieved object data to the second database management system, the second database being a database with a single table;
• and
• receiving the retrieved object data, processing the particular set of property values in the retrieved object data, and writing all of the property values in the particular set of property values to the single table of the second database, whereby the second database is updated to reflect changes in the first database.