System and methods for robust synchronization of datasets

A method of synchronizing records of a client dataset with records of a reference dataset of a synchronization engine, the method comprising the steps of:

• A) identifying a first plurality of changes to the records in the client dataset to be synchronized with the records of the reference dataset;
• B) initiating a synchronization session between the client dataset and the reference dataset;
• and
• C) communicating said first plurality of changes to the reference dataset and synchronizing said first plurality of changes into the records of the reference dataset, said step of communicating and synchronizing the first plurality of changes further comprising the steps of:
• a) sending a subset of the first plurality of changes from the client dataset to the synchronization engine;
• b) receiving the subset of the first plurality of changes at the synchronization engine;
• c) synchronizing the subset of the first plurality of changes into the reference dataset and making the subset of the first plurality of changes available in said reference dataset for user access and for synchronization with another dataset before all of said first plurality of changes are received by said synchronization engine;
• d) sending acknowledgements of the receipt of the subset of the first plurality of changes from the synchronization engine;
• e) receiving the acknowledgements of the receipt of the subset of the first plurality of changes at the client dataset;
• f) repeating steps a) to e) until all of the first plurality of changes are received at the synchronization engine, synchronized into the reference dataset and acknowledged back to the client dataset;
• and
• g) if said synchronization session fails before all of said first plurality of changes are received by said synchronization engine, synchronized into the reference dataset and acknowledged back to the client dataset, initiating a subsequent synchronization session and continuing the step C), without resending from the client dataset any changes for which acknowledgements have already been received at the client dataset.

A method of synchronizing a first dataset with a second dataset, each of said first and second datasets comprising a plurality of corresponding records, wherein changes have been made to the records in the first dataset that have not yet been synchronized with the second dataset, the method comprising:

• identifying a plurality of changes to the records in the first dataset to be synchronized with the records in the second dataset in a synchronization session, wherein said plurality of changes have not yet been synchronized with the second dataset;
• initiating a synchronization session to synchronize said plurality of changes from the first dataset to the second dataset;
• sending a subset of one or more of said plurality of changes, but not all of said plurality of changes, from said first dataset to said second dataset;
• receiving said subset of changes at said second dataset;
• updating a last synchronization time for each of the records affected by the subset of the plurality of changes;
• propagating the subset of changes into said second dataset;
• and
• if there is no failure of said synchronization session, completing said synchronization by sending the remainder of said plurality of changes from said first dataset to said second dataset, propagating the remainder of said plurality of changes into said second dataset and updating an overall last synchronization time;
• or, if there is a failure of said synchronization session before the remainder of the plurality of changes are received by said second dataset and propagated into said second dataset, not updating the overall last synchronization time,
• wherein the last synchronization times are used to determine changes that should be synchronized in subsequent synchronization sessions.