Methods for estimating cost savings using deduplication and compression in a storage system

A method for estimation, comprising:

• accessing, over a communication network, data from a unit of storage of a data storage system;
• parsing a plurality of blocks from the data, wherein each of the plurality of blocks of data is uncompressed;
• determining a compression ratio for each of the blocks;
• for each of the blocks, assigning the block to one of a plurality of buckets based on the determined compression ratio for the block, wherein the plurality of buckets are associated with respective ranges of compression ratios;
• generating a plurality of fingerprints based on the plurality of blocks using a hash algorithm;
• estimating a cardinality of each of the plurality of buckets, each cardinality representing an estimated number of unique blocks for a respective one of the buckets;
• and
• estimating a post-deduplication compression ratio for the plurality of blocks stored in the unit of storage based on the respective ranges of compression ratios associated with each of the buckets and the estimated cardinality of each of the buckets.

A non-transitory computer-readable medium comprising program instructions to:

• access, over a communication network, data from a unit of storage of a data storage system;
• parse a plurality of blocks from the data, wherein each of the plurality of blocks of data is uncompressed;
• determine a compression ratio for each of the blocks;
• for each of the blocks, assign the block to one of a plurality of buckets based on the determined compression ratio for the block, wherein the plurality of buckets are associated with respective ranges of compression ratios;
• estimate a cardinality of each of the plurality of buckets, each cardinality representing an estimated number of unique blocks for a respective one of the buckets;
• and
• estimate a post-deduplication compression ratio for the plurality of blocks stored in the unit of storage based on the respective ranges of compression ratios associated with each of the buckets and the estimated cardinality of each of the buckets.