Reducing digest storage consumption by tracking similarity elements in a data deduplication system

A system for reducing digests storage consumption by tracking numbers of similarity elements in a similarity search structure in a data deduplication system of a computing environment, the system comprising:

• the data deduplication system;
• a repository operating in the data deduplication system;
• a similarity search structure in association with the repository in the data deduplication system;
• and
• at least one processor device operable in the computing storage environment for controlling the data deduplication system, wherein the at least one processor device:
• partitions input data into chunks and grouping the chunks into chunk sets,
• calculates digests for the input data and storing the digests in sets corresponding to the chunk sets,
• calculates similarity elements for the input data and storing the similarity elements in the similarity search structure,
• maintains for each one of the chunk sets a number of the similarity elements associated with the chunk set which are currently contained in the similarity search structure, and
• removes a digests set associated with that chunk set from the repository when a number of specific chunk sets becomes lower than a predetermined threshold.

A computer program product for reducing digests storage consumption by tracking the numbers of similarity elements in a similarity search structure in a data deduplication system using a processor device in a computing environment, the computer program product comprising a computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

• a first executable portion that partitions input data into chunks and grouping the chunks into chunk sets;
• a second executable portion that calculates digests for the input data and storing the digests in sets corresponding to the chunk sets;
• a third executable portion that calculates similarity elements for the input data and storing the similarity elements in a similarity search structure;
• a fourth executable portion that maintains for each one of the chunk sets a number of the similarity elements associated with the chunk set which are currently contained in the similarity search structure;
• and
• a fifth executable portion that removes a digests set associated with a chunk set from the repository when the number of similarity elements of that chunk set becomes lower than a threshold.