Continuous real-time masked database replication

A computer readable non-transitory storage medium, storing instructions that when executed by a computer processor cause the computer processor to perform steps comprising:

• receiving one or more change logs, each corresponding to a respective update event made to data of the source database;
• masking the one or more change logs using a predetermined masking algorithm to generate a set of masked change logs;
• storing the set of masked change logs in a pre-buffer;
• responsive to a determination that the set of masked change logs stored in the pre-buffer corresponds to a complete transaction:
• determining whether any conflicts exist between any of the set of masked change logs stored in the pre-buffer and one or more masked change logs stored in a main buffer, wherein a conflict exists when a change log of the set of masked change logs and a change log of the one or more masked change logs stored in the main buffer are configured to operate on the same data of the masked replica database;
• responsive to determining that a conflict exists between any of the set of masked change logs stored in the pre-buffer and the one or more masked change logs stored in the main buffer, applying the one or more masked change logs stored in the main buffer to the masked replica database to update the masked replica database, and flushing the main buffer;
• moving the set of masked change logs corresponding to a complete transaction from the pre-buffer to the main buffer.

A computer system comprising:

• a computer processor;
• and
• non-transitory computer-readable storage medium storing instructions for executing by the computer processor, the instructions for:
• receiving one or more change logs, each corresponding to a respective update made to data of the source database;
• masking the one or more change logs using a predetermined masking algorithm to generate one or more a set of masked change logs;
• storing the one or more set of masked change logs in a pre-buffer configured to store a set of masked change logs;
• responsive to a determination that the set of masked change logs stored in the pre-buffer corresponds to a complete transaction:
• determining whether any conflicts exist between any of the set of masked change logs stored in the pre-buffer and one or more masked change logs stored in a main buffer, wherein a conflict exists when a change log of the set of masked change logs and a change log of the one or more masked change logs stored in the main buffer are configured to operate on the same data of the masked replica database;
• responsive to determining that a conflict exists between any of the set of masked change logs stored in the pre-buffer and the one or more masked change logs stored in the main buffer, applying the one or more masked change logs stored in the main buffer to the masked replica database to update the masked replica database, and flushing the main buffer;
• moving the set of masked change logs corresponding to a complete transaction from the pre-buffer to the main buffer.