Identifying duplicate segments and the latest segment using a live instance filter

A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for managing data in a storage system, the method comprising:

• selecting a suffixed fingerprint from a bucket;
• making a first determination that the suffixed fingerprint is live using a live filter;
• based on the first determining, making a second determination that a container identification number (CID) associated with the suffixed fingerprint is higher than a cached fingerprint in a live instance filter;
• based on the second determining, replacing the cached fingerprint in the live instance filter with the suffixed fingerprint;
• and
• marking everything not in the live instance filter as available for garbage collection.

A system for managing data in a storage system, the system comprising:

• a processor comprising circuitry;
• memory comprising instructions, which when executed by the processor perform a method, the method comprising:
• selecting a suffixed fingerprint from a bucket;
• making a first determination that the suffixed fingerprint is live using a live filter;
• based on the first determining, making a second determination that a container identification number (CID) associated with the suffixed fingerprint is higher than a cached fingerprint in a live instance filter;
• based on the second determining, replacing the cached fingerprint in the live instance filter with the suffixed fingerprint;
• and
• marking everything not in the live instance filter as available for garbage collect.