System and method for incremental replication
AssigneePalantir Technologies Inc.
InventorsAlexander Landau, Tim Wilson, Richard Allen Ducott, III
Life of this patent
Estimating the term…
Abstract
A method of incrementally replicating investigative analysis data is disclosed along with a system for performing the same. The method and system provide the ability to break a data replication job into multiple “replication chunks” which can be exported and imported separately. By doing so, the method and system can efficiently replicate large numbers of data object changes over an unreliable data network.
Claims18
A method performed by a computing system comprising one or more processors and storage media storing one or more programs, the one or more programs comprising instructions executed by the one or more processors to perform the method, the method comprising:
- receiving, at an importing system, from an exporting system, a replication chunk of an incremental replication job, wherein the replication chunk contains a job identifier and a chunk identifier;
- performing causality detection between change data in the replication chunk and existing change data at the importing system;
- and
- based at least in part on results of the causality detection, importing the replication chunk into the importing system by:
- (a) superseding the existing change data already at the importing system with the change data in the replication chunk, (b) not superseding the existing change data already at the importing system with the change data in the replication chunk, or (c) determining that a concurrency conflict exists between the change data in the replication chunk and the existing change data at the importing system.
One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, causes performance of a method comprising the steps of:
- receiving, at an importing system, from an exporting system, a replication chunk of an incremental replication job, wherein the replication chunk contains a job identifier and a chunk identifier;
- performing causality detection between change data in the replication chunk and existing change data already stored at the importing system;
- and
- based at least in part on results of the causality detection, importing the replication chunk into the importing system by:
- (a) superseding, at the importing system, the existing change data already stored at the importing system with the change data in the replication chunk, (b) not superseding, at the importing system, the existing change data already stored at the importing system with the change data in the replication chunk, or (c) determining that a concurrency conflict exists between the change data in the replication chunk and the existing change data already stored at the importing system.
A system comprising:
- a database;
- an importing system comprising one or more computer-readable media and one or more processors, the one or more computer-readable media storing instructions which, when executed by the one or more processors, causes performance of a method comprising the steps of:
- receiving, at an importing system, from an exporting system, a replication chunk of an incremental replication job, wherein the replication chunk contains a job identifier and a chunk identifier;
- performing causality detection between change data in the replication chunk and existing change data at the importing system;
- and
- based at least in part on results of the causality detection, importing the replication chunk into the importing system by:
- (a) superseding, at the importing system, the existing change data already stored at the importing system with the change data in the replication chunk, (b) not superseding, at the importing system, the existing change data already stored at the importing system with the change data in the replication chunk, or (c) determining that a concurrency conflict exists between the change data in the replication chunk and the existing change data already stored at the importing system.
Dependencies are inferred from the claim text.