# US 2019/0171650 A1 · System and method to improve data synchronization and integration of heterogeneous databases distributed across enterprise and cloud using bi-directional transactional bus of asynchronous change data system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171650 A1
- Document ID: US20190171650A1
- Status: Published application
- Kind: A1
- Application number: 16/199,269
- Filed: 2018-11-26
- Published: 2019-06-06
- Earliest claimed: 2017-12-01
- Inventors: Chavdar Botev; Rajeev Bharadhwaj; Burton Hipp
- Web page: https://patentgrep.com/patent/US20190171650A1

## CPC

- G06F 16/273: {Asynchronous replication or reconciliation}
- G06F 16/128: {Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion (error detection or correction of the data by redundancy in operations or in hardware G06F11/14, G06F11/16)}
- G06F 16/1805: {Append-only file systems, e.g. using logs or journals to store data}
- G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
- G06F 16/2358: {Change logging, detection, and notification (replication G06F16/27)}
- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06F 16/2386: {Bulk updating operations (data conversion details G06F16/258)}
- G06F 16/256: {in federated or virtual databases}
- G06F 16/289: {Object oriented databases}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Disclosed are a method and/or a system to improve data synchronization and integration of heterogeneous databases distributed across enterprise and/or cloud using bi-directional transactional bus of asynchronous change data system. In one embodiment, a method of snapshot materialization and application consistency includes running a change capture system to capture all changes by collecting a change capture data, running an initial bulk load of all data in a source system, and applying all change transactions to a particular transaction id. The method includes removing a reappearance of a record using keys that handle de-duplication of entries and deeming a snapshot of a target system. The change capture data concerns in the source system in an order of its occurrence. A logical clock value determines the order in which the changes have occurred. The changes are a transactional and/or a non-transactional. The transaction boundaries are preserved as part of the change capture data.

## Claims

### Claim 1

A method of snapshot materialization and application consistency, comprising:
running a change capture system to capture all changes by first collecting a change capture data before an initial load is started,
wherein the change capture data concerns modifications including an insert, a delete, and an update in a source system in an order of its occurrence,
wherein a logical clock value determines the order in which the changes have occurred,
wherein the changes are at least one of a transactional and a non-transactional and
wherein the transaction boundaries are preserved as part of the change capture data;
running a initial bulk load of all data in the source system while change capture is progressing;
applying all change transactions to a particular transaction id when the initial load is completed;
removing a reappearance of a record using keys that handle de-duplication of entries; and
deeming a snapshot of a target system as consistent if the state of the whole system is identical to a source database at a certain transaction id,
wherein a historic consistent snapshot with past data is preserved for at east one of a historic data analysis, an auditing, and a backup,
wherein the key is at least one of a primary key and a composite key,
wherein the source system is at least one of a OLTP RDBMS, a NoSQL database, an API system, and a message bus,
wherein the source system to run on at least one of a bare metal hardware, a VM, a private, and a public cloud,
wherein the target system is at least one of a OLTP RDBMS, a OLAP Data Warehouse, a Data Lake, and a NoSQL database
wherein the target system to run on at least one of a bare metal hardware, a VM, a private or public cloud,
wherein the target system to utilize at least one of a row storage, a columnar storage, an in-place update storage, and an append-only storage,
wherein the target system to support a transactional update,
wherein the target system is at least one of an identical and a different type than the source system,
wherein a hierarchical declarative replication policy is applied to the change capture data and initial load data to filter, mask and modify the source data to be used among others for synchronizing a subset of the source data, remove and mask sensitive and personal data, transform the source data format and types to match the target system,
wherein a target database is at least one of a RDBMS in-place update and HDFS append-only database, and
wherein a replicated table in the target database is presented as at least one of an object materialized view, database view, and a database materialized view.

### Claim 2 (depends on 1)

The method of claim 1 wherein the target system to not support the transactional update.
