Cloud capacity scaling with high readable capacity in metadata space constrained deduplication systems

A system for scaling capacity in a deduplicated storage appliance having limited metadata space, the system comprising:

• a processor;
• and memory configured to store one or more sequences of instructions which, when executed by the processor, cause the processor to carry out the steps of:
• maintaining a plurality of cloud units in a cloud storage, each cloud unit comprising a plurality of containers comprising data containers storing segments of files, segment tree containers storing upper-level segments of segment trees representing the files, and cloud containers storing headers from the data and segment tree containers, wherein a header for a data container comprises fingerprints identifying the segments of the files, and a header for a segment tree container comprises fingerprints identifying the upper-level segments;
• storing, at local storage of the deduplicated storage appliance, a first level of metadata for each cloud unit in a read-write state, a second level of metadata for each cloud unit in a read-only state, and a third level of metadata for each cloud unit in an offline state, the third level of metadata requiring less storage space than the first and second levels of metadata, and the second level of metadata requiring less storage space than the first level of metadata;
• and
• managing the limited metadata space on the deduplicated storage appliance by maintaining at least a subset of the plurality of cloud units in the read-only state.

A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method of scaling capacity in a deduplicated storage appliance having limited metadata space comprising:

• maintaining a plurality of cloud units in a cloud storage, each cloud unit comprising a plurality of containers comprising data containers storing segments of files, segment tree containers storing upper-level segments of segment trees representing the files, and cloud containers storing headers from the data and segment tree containers, wherein a header for a data container comprises fingerprints identifying the segments of the files, and a header for a segment tree container comprises fingerprints identifying the upper-level segments;
• storing, at local storage of the deduplicated storage appliance, a first level of metadata for each cloud unit in a read-write state, a second level of metadata for each cloud unit in a read-only state, and a third level of metadata for each cloud unit in an offline state, the third level of metadata requiring less storage space than the first and second levels of metadata, and the second level of metadata requiring less storage space than the first level of metadata;
• and
• managing the limited metadata space on the deduplicated storage appliance by maintaining at least a subset of the plurality of cloud units in the read-only state.