Region-integrated data deduplication implementing a multi-lifetime duplicate finder

A deduplicating storage system configured to perform deduplication in conjunction with random read and write operations across a namespace, the system comprising:

• a processor and logic integrated with the processor, executable by the processor, or integrated with and executable by the processor, the logic being configured to cause the processor to perform a method comprising:
• computing a fingerprint of a data chunk;
• determining whether a short term dictionary corresponding to the namespace comprises an entry corresponding to the fingerprint;
• in response to determining the short term dictionary comprises the entry corresponding to the fingerprint, writing the data chunk to a data store corresponding to the namespace in a deduplicating manner;
• in response to determining the short term dictionary does not comprise the entry corresponding to the fingerprint, determining whether a long term dictionary corresponding to the namespace comprises the entry corresponding to the fingerprint;
• in response to determining the long term dictionary comprises the entry corresponding to the fingerprint:
• writing the data chunk to the data store in the deduplicating manner;
• and
• repopulating the short term dictionary with the entry corresponding to the fingerprint;
• and
• in response to determining the long term dictionary does not comprise the entry corresponding to the fingerprint, writing the data chunk to the data store in a non-deduplicating manner,
• wherein the short term dictionary comprises a first eviction policy,
• wherein the long term dictionary comprises a second eviction policy,
• wherein the first eviction policy is configured to evict one or more entries of the short term dictionary in response to a new entry being inserted into the short term dictionary, and
• wherein the second eviction policy is configured to evict one or more entries of the long term dictionary in response to a new entry being inserted into the long term dictionary.