Database synchronization using resizable invertible bloom filters with database snapshots

A non-transitory computer-readable storage medium storing executable computer instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the instructions comprising instructions to:

• generate a first resizable invertible bloom filter of a source data table based on information of the source data table at a first point in time, the first resizable invertible bloom filter having a first size that is determined based on an estimated number of changes in the source data table between the first point in time and a second point in time;
• generate a second resizable invertible bloom filter for the source data table based on information of the source data table at the second point in time, the second resizable invertible bloom filter also having the first size;
• determine an actual number of changes in the source data table between the first point in time and the second point in time;
• responsive to the actual number of changes being smaller than the estimated number of changes, determine a second size for the first and the second resizable invertible bloom filters, the second size being smaller than the first size;
• update the first resizable invertible bloom filter and the second resizable invertible bloom filter to have the second size;
• identify changes in the source data table that occurred between the first point in time and the second point in time based on the updated first resizable invertible bloom filter and the updated second resizable invertible bloom filter;
• and
• send instructions to a destination data table, the instructions comprising information to perform an operation that synchronizes the destination data table with the source data table based on the identified changes.

A computing system comprising:

• a processor;
• and
• a non-transitory computer-readable storage medium storing instructions, the instructions when executed by the processor cause the processor to perform steps including:
• generating a first resizable invertible bloom filter of a source data table based on information of the source data table at a first point in time, the first resizable invertible bloom filter having a first size that is determined based on an estimated number of changes in the source data table between the first point in time and a second point in time;
• generating a second resizable invertible bloom filter for the source data table based on information of the source data table at the second point in time, the second resizable invertible bloom filter also having the first size;
• determining an actual number of changes in the source data table between the first point in time and the second point in time;
• responsive to the actual number of changes being smaller than the estimated number of changes, determining a second size for the first and the second resizable invertible bloom filters, the second size being smaller than the first size;
• updating the first resizable invertible bloom filter and the second resizable invertible bloom filter to have the second size;
• identifying changes in the source data table that occurred between the first point in time and the second point in time based on the updated first resizable invertible bloom filter and the updated second resizable invertible bloom filter;
• and
• sending instructions to a destination data table, the instructions comprising information to perform an operation that synchronizes the destination data table with the source data table based on the identified changes.