Identifying changes between source and destination databases using snapshots of source database and invertible bloom filters

A non-transitory computer-readable storage medium storing executable computer instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the instructions comprising instructions to:

• retrieve a source data table and a destination data table;
• transform each row of data in the source data table into a source row representation;
• transform each row of data in the destination data table into a destination row representation;
• generate a first invertible bloom filter for the source data table using the respective source row representations and a second invertible bloom filter for the destination data table using the respective destination row representations, wherein generating the first invertible bloom filter and the second invertible bloom filter comprises:
• determining a size for both the first invertible bloom filter and the second invertible bloom filter based on an estimated number of changes to the source data table, wherein a size of an invertible bloom filter is associated with a space occupied in a computer memory,
• determining a difference between the estimated number of changes and an actual number of changes to the source data table, and
• adjusting the size responsive to the determined difference meeting a threshold;
• generate a third invertible bloom filter by subtracting the second invertible bloom filter from the first invertible bloom filter;
• identify a difference between a row of data in the source data table and a row of data in the destination data table by decoding the third invertible bloom filter;
• and
• transmit information associated with the identified difference to synchronize the destination data table with the source data table by updating the row of data in the destination data table based on the row of data in the source data table.

A computing system comprising:

• one or more processors;
• and
• a non-transitory computer-readable storage medium storing instructions for database synchronization, the instructions when executed by the processors cause the processors to perform operations including:
• retrieving a source data table and a destination data table;
• transforming each row of data in the source data table into a source row representation;
• transforming each row of data in the destination data table into a destination row representation;
• generating a first invertible bloom filter for the source data table using the respective source row representations and a second invertible bloom filter for the destination data table using the respective destination row representations, wherein generating the first invertible bloom filter and the second invertible bloom filter comprises:
• determining a size for both the first invertible bloom filter and the second invertible bloom filter based on an estimated number of changes to the source data table, wherein a size of an invertible bloom filter is associated with a space occupied in a computer memory,
• determining a difference between the estimated number of changes and an actual number of changes to the source data table, and
• adjusting the size responsive to the determined difference meeting a threshold;
• generating a third invertible bloom filter by subtracting the second invertible bloom filter from the first invertible bloom filter;
• identifying a difference between a row of data in the source data table and a row of data in the destination data table by decoding the third invertible bloom filter;
• and
• transmitting information associated with the identified difference to synchronize the destination data table with the source data table by updating the row of data in the destination data table based on the row of data in the source data table.