Scalable garbage collection for deduplicated storage
AssigneeEMC IP Holding Company LLC
InventorsPhilip Shilane, Kimberly Lu, Joseph Brandt, Nicholas Noto, Tipper Truong, Mariah Arevalo
Life of this patent
Estimating the term…
Abstract
Systems and methods for cleaning a storage system. A deduplicated storage system is cleaned by identifying structures that include dead or unreferenced segments. This includes processing recipes to identify the segments that are no longer part of a live object recipe. Then, the dead segments are removed. This is accomplished by copying forward the live segments and then deleting, as a whole, the structure that included the dead segments.
Claims20
A method for cleaning deleted objects from a storage system, the method comprising:
- creating a plurality of workers to perform a garbage collection operation that cleans the deleted objects stored in the storage system from objects stored in the storage system;
- identifying impacted similarity groups that are impacted by the garbage collection operation, wherein the impacted similarity groups include segments associated with the deleted objects and segments associated with live objects;
- identifying live segments in the impacted similarity groups associated with the live objects;
- removing the segments associated with the deleted objects;
- updating the impacted similarity groups to reflect the segments associated with the deleted objects have been removed.
A method for cleaning deleted objects from a deduplicated storage system, wherein the deduplicated storage system stores objects using an object recipe that references slice recipes, wherein the slice recipes reference similarity groups and wherein the similarity groups reference compression regions, wherein segments of objects in the storage system are stored in the compression regions, the method comprising:
- performing a startup phase of a garbage collection operation in order to instantiate a controller, wherein the garbage collection operation cleans unreferenced segments from the storage system;
- estimating a number of workers, by the controller, needed to perform the garbage collection operation based on a number of impacted similarity groups and worker capacity;
- determining impacted similarity groups in the storage system that are associated with the unreferenced segments, wherein the impacted similarity groups are associated with the unreferenced segments and live segments;
- marking the live segments in the impacted similarity groups;
- removing the unreferenced segments;
- and
- updating the impacted similarity groups to reflect the unreferenced segments associated with the deleted objects have been removed.
Dependencies are inferred from the claim text.