Watermark-based techniques for change-data-capture

One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

• selecting, from a datastore, a chunk comprising a plurality of rows of data;
• comparing the plurality of rows of data in the chunk with a first plurality of log events that are associated with the chunk and included in a change log associated with the datastore;
• determining that a first row of data included in the chunk conflicts with the first plurality of log events;
• removing the first row of data from the chunk;
• and
• transmitting, to an output, one or more log events associated with one or more rows of data remaining in the chunk.

A system, comprising:

• one or more processors;
• and
• one or more memories coupled to the one or more processors and including one or more instructions that, when executed by the one or more processors, cause the one or more processors to:
• select, from a datastore, a chunk comprising a plurality of rows of data;
• compare the plurality of rows of data in the chunk with a first plurality of log events that are associated with the chunk and included in a change log associated with the datastore;
• determine that a first row of data included in the chunk conflicts with the first plurality of log events;
• remove the first row of data from the chunk;
• and
• transmit, to an output, one or more log events associated with one or more rows of data remaining in the chunk.