Digest retrieval based on similarity search in data deduplication

A system for retrieving digests based on a similarity search for efficient deduplication processing in a data deduplication system of a computing environment, the system comprising:

• the data deduplication system;
• a repository operating in the data deduplication system;
• a memory in the data deduplication system;
• a search structure in association with the memory in the data deduplication system;
• and
• at least one processor device operable in the computing storage environment for controlling the data deduplication system, wherein the at least one processor device:
• partitions input data into data chunks,
• calculates for each of the data chunks similarity elements and digest values,
• searches for matching similarity elements in the search structure containing similarity elements,
• finds positions of similar data in a repository of data,
• uses the positions of the similar data to locate and load into the memory stored digest values of the similar data in the repository, and
• matches the digest values of the input data with the stored digest values loaded into memory to find data matches.

A computer program product for retrieving digests based on a similarity search for efficient deduplication processing in a data deduplication system using a processor device in a computing environment, the computer program product comprising a computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

• a first executable portion that partitions input data into data chunks;
• a second executable portion that calculates for each of the data chunks similarity elements and digest values;
• a third executable portion that searches for matching similarity elements in the search structure containing similarity elements;
• a fourth executable portion that finds positions of similar data in a repository of data;
• a fifth executable portion that uses the positions of the similar data to locate and load into the memory stored digest values of the similar data in the repository;
• and
• a sixth executable portion that matches the digest values of the input data with the stored digest values loaded into memory to find data matches.