Automatic archiving of data store log data
AssigneeAmazon Technologies, Inc.
InventorLei Ye
Life of this patent
Estimating the term…
Abstract
Methods, systems, and computer-readable media for automatic archiving of data store log data are disclosed. One or more operation records in a log are selected for archival. The one or more operation records comprise data indicative of operations performed on one or more data objects of a data store. The one or more operation records are selected for archival prior to deletion from the log. The one or more operation records are replicated from the log to an archive. Based at least in part on the replicating, the one or more operation records in the log are marked as archived. Based at least in part on the marking as archived, the deletion of the one or more operation records from the log is permitted.
Claims20
A system, comprising:
- one or more computing devices configured to implement a data store archiving system, wherein the data store archiving system is configured to:
- select, for archival, one or more operation records in a log, wherein the one or more operation records comprise data indicative of operations performed on one or more data objects of a data store, and wherein the one or more operation records are selected for archival prior to deletion from the log;
- assign, to one or more workers, one or more archiving jobs comprising data indicative of the one or more operation records selected for archival;
- flag the one or more operation records as unarchived;
- postpone the deletion of the one or more operation records from the log based at least in part on the flagging the one or more operation records as unarchived;
- replicate, subsequent to flagging the one or more operation records as unarchived, the one or more operation records from the log to an archive, wherein the one or more operation records are replicated from the log to the archive by the one or more workers;
- generate metadata indicative of a mapping between one or more of the data objects referenced in the one or more operation records and one or more locations of the one or more operation records in the archive;
- and
- mark the one or more operation records for deletion from the log after the one or more operation records are replicated to the archive.
A computer-implemented method, comprising:
- selecting, for archival, one or more operation records in a log, wherein the one or more operation records comprise data indicative of operations performed on one or more data objects of a data store, and wherein the one or more operation records are selected for archival prior to deletion from the log;
- marking the one or more operation records as unarchived;
- postponing the deletion of the one or more operation records from the log based at least in part on the marking the one or more operation records as unarchived;
- replicating, subsequent to marking the one or more operation records as unarchived, the one or more operation records from the log to an archive;
- marking the one or more operation records in the log as archived based at least in part on the replicating the one or more operation records from the log to the archive;
- and
- permitting deletion of the one or more operation records from the log based at least in part on the marking the one or more operation records in the log as archived.
One or more non-transitory computer-readable storage media storing program instructions computer-executable on or across one or more processors to perform:
- selecting, for archival, one or more operation records in a log, wherein the one or more operation records comprise data indicative of operations performed on one or more data objects of a data store, and wherein the one or more operation records are selected for archival prior to deletion from the log;
- marking the one or more operation records as unarchived;
- postponing the deletion of the one or more operation records from the log based at least in part on the marking the one or more operation records as unarchived;
- replicating, subsequent to marking the one or more operation records as unarchived, the one or more operation records from the log to an archive;
- generating metadata indicative of a mapping between one or more of the data objects referenced in the one or more operation records and one or more locations of the one or more operation records in the archive;
- and
- marking the one or more operation records in the log as archived based at least in part on the replicating the one or more operation records from the log to the archive;
- and
- causing deletion of the one or more operation records from the log based at least in part on the marking the one or more operation records in the log as archived.
Dependencies are inferred from the claim text.