Back to results
US 11,386,060 B1Granted
Techniques for verifiably processing data in distributed computing systems
AssigneeAmazon Technologies, Inc.
InventorsRyan Charles Schmitt, Nguyen Nguyen, Jason Neil Yakimovich
Life of this patent
Estimating the term…
Abstract
A system implements techniques for processing data across multiple layers. In some embodiments, a distributed data processing system tracks processing states via a monotonic process, whereby each state is irreversibly entered and cannot be entered without a predecessor state being verifiably completed. In some of such embodiments, a multi-layer data storage system tracks and processes data deletions across disparate layers such that a deletion state can be positively deduced from the deletion states of each of the disparate layers.
Claims20
A computer-implemented method, comprising:
- in response to receiving deletion requests, recording, in a table of a deletion store, deletion information for data associated with the deletion requests for deletion, the data being stored on a plurality of disparate storage layers, the plurality of disparate storage layers including at least preliminary storage in a first storage layer and durable storage in a second storage layer, the durable storage including a plurality of volumes;
- sorting the deletion requests so as to group the deletion requests into volume groups, the volume groups associated with one or more volumes of the plurality of volumes on which the data is stored;
- and
- processing, for each volume group of the volume groups using a monotonic deletion routine so as to delete the data of a respective volume group from the plurality of disparate storage layers, the monotonic deletion routine comprising:
- deleting the data of the associated volume group from the preliminary storage;
- verifying that the data of the respective volume group is deleted from the preliminary storage;
- as a result of verifying that the data of the respective volume group is deleted from the preliminary storage, waiting for a determined period;
- as a result of waiting for the determined period, deleting the data of the associated volume group from a respective volume associated with the respective volume group of the plurality of volumes;
- verifying that the data of the respective volume group is deleted from the respective volume;
- as a result of verifying that the data of the respective volume group is deleted from the respective volume, reclaiming space on the respective volume;
- verifying that the space on the respective volume is reclaimed;
- as a result of verifying that the space on the respective volume is reclaimed, removing the deletion information for the data of the respective volume group from the table;
- verifying that the deletion information is removed from the table;
- and
- as a result of verifying that the deletion information is removed, confirming that the monotonic deletion routine is complete.
12 more elements
A system, comprising:
- at least one computing device configured to implement one or more services, wherein the one or more services are configured to:
- sort a plurality of deletion requests in a deletion table into volume groups associated with one or more volumes of a plurality of storage layers associated with the system;
- query the deletion table for information associated with data to be deleted from the plurality of storage layers associated with the system;
- for each volume group of the volume groups, use a monotonic deletion routine to delete the data of a respective volume group from the plurality of storage layers, the monotonic deletion routine comprising causing a first storage layer of the plurality of storage layers to delete the data from the first storage layer;
- as a result of verifying that the data is deleted from the first storage layer, wait for a determined period;
- as a result of waiting for the determined period, cause a second storage layer to delete the data from one or more volumes of the second storage layer;
- as a result of verifying that the data is deleted from the second storage layer, cause the storage layer to reclaim space on the one or more volumes;
- as a result of verifying that the space on the one or more volumes is reclaimed, cause the removal of the information from the deletion table;
- and
- as a result of verifying that the deletion information is removed, confirm to at least one requester that the data is deleted.
7 more elements
A non-transitory computer-readable storage medium having stored thereon executable instructions that, when executed by one or more processors of a computer system, cause the computer system to at least:
- sort a plurality of deletion requests into volume groups associated with one or more volumes of a plurality of storage layers associated with the computer system;
- and
- for each volume group of the volume groups, use a monotonic deletion routine to delete data of a respective volume group from the plurality of storage layers, the monotonic deletion routine comprising causing the system to delete the data stored on the plurality of storage layers associated with the computer system by at least:
- removing the data from a first storage layer of the plurality of storage layers;
- as a result of verifying that the data is removed from the first storage layer, waiting for a determined period;
- as a result of waiting for the determined period, removing the data from a second storage layer of the plurality of storage layers;
- as a result of verifying that the data is removed from the second storage layer, reclaiming space on the second storage layer;
- as a result of verifying that the space on the second storage layer is reclaimed, removing, from a deletion table that tracks deleted data on the plurality of storage layers, information related to the data;
- and
- as a result of verifying that the deletion information is removed, confirming to a requester that the data is deleted.
7 more elements
Dependencies are inferred from the claim text.