Inline deduplication using log based storage

A system for use in managing deduplication of data in storage systems, the system comprising a processor configured to:

• identify a candidate data object for deduplicating a data object by evaluating digests stored in a current digest segment to determine whether another digest matching a digest associated with a data block is stored in the current digest segment, wherein the current digest segment includes a set of digests associated with a set of data blocks previously received for deduplication, wherein each of the set of digests is determined to be unique prior to being stored in the current digest segment in an order associated with the set of data blocks previously received for deduplication;
• based on the evaluation, apply a deduplicating technique to the data object;
• store the current digest segment in an index table, wherein a previous digest segment associated with a digest stored in the index table matches the digest associated with the data block is replaced by the current digest segment;
• and
• organize a plurality of digest segments into a segment group and associating a reference counter with the segment group, wherein the reference counter tracks temporal characteristics of the segment group, wherein if the reference counter reaches zero, storage space consumed by the digest group is reclaimed.