Data equivalency system using co-pair manifests for identifying duplicates in generations of resources

A computer-implemented method comprising:

• generating a listing of equivalent snapshot objects that are referenced by one or more of a set of object-based snapshots, each object-based snapshot representing data of a corresponding block-storage volume at a point-in-time as a collection of snapshot objects that provide a copy of data of the block-storage volume at the point-in-time, wherein generating the listing comprises:
• retrieving from an object data store a set of main manifests corresponding to the set of object-based snapshots, each main manifest including a full listing of snapshot objects within the object data store that collectively represent data of the corresponding block-storage volume at the point-in-time of the object-based snapshot;
• retrieving from the object data store a set of co-pair manifests corresponding to the set of object-based snapshots, each co-pair manifest corresponding to an object-based snapshot, and including a listing of encrypted snapshot objects with equivalencies to unencrypted snapshot objects, wherein the encrypted snapshot objects are distinct from the unencrypted snapshot objects, and wherein an equivalency indicates that an encrypted snapshot object represents duplicate storage, in encrypted form, as block-storage volume source data stored in an unencrypted form as an unencrypted snapshot object;
• identifying data duplication in the set of main manifests based on listings of encrypted snapshot objects with equivalencies to unencrypted snapshot objects found in the set of co-pair manifests;
• and
• removing identified data duplication in the set of main manifests.

One or more non-transitory computer-readable media comprising instructions executable by one or more processors to generate a listing of equivalent snapshot objects that are referenced by one or more of a set of object-based snapshots, each object-based snapshot representing data of a corresponding block-storage volume at a point-in-time as a collection of snapshot objects that provide a copy of data of the block-storage volume at the point-in-time, wherein the instructions, when executed, cause the one or more processors to:

• retrieve from an object data store a set of main manifests corresponding to the set of object-based snapshots, each main manifest including a full listing of snapshot objects within the object data store that collectively represent data of the corresponding block-storage volume at the point-in-time of the object-based snapshot;
• retrieve from the object data store a set of co-pair manifests corresponding to the set of object-based snapshots, each co-pair manifest corresponding to an object-based snapshot, and including a listing of encrypted snapshot objects with equivalencies to unencrypted snapshot objects, wherein the encrypted snapshot objects are distinct from the unencrypted snapshot objects and wherein an equivalency indicates that an encrypted snapshot object represents duplicate storage, in encrypted form, as block-storage volume source data stored in unencrypted form as an unencrypted snapshot object;
• identify data duplication in the set of main manifests based on listings of encrypted snapshot objects with equivalencies to unencrypted snapshot objects found in the set of co-pair manifests;
• and
• remove identified data duplication in the set of main manifests.