Method and system of similarity-based deduplication

A method for similarity-based deduplication, the method comprising:

• receiving, by a processor, an input data block;
• computing, by the processor, discrete wavelet transform (DWT) coefficients based on the input data block;
• extracting, by the processor, feature-related DWT data from the computed DWT coefficients;
• applying, by the processor, quantization to the extracted feature-related DWT data to obtain keys as results of the quantization;
• constructing, by the processor, a locality-sensitive fingerprint of the input data block based on the keys;
• computing, by the processor, a similarity degree between the locality-sensitive fingerprint of the input data block and a locality-sensitive fingerprint of each data block of a plurality of data blocks in a cache memory;
• selecting, by the processor, an optimal reference data block as the data block from the plurality of data blocks that has a largest similarity degree with the input data block;
• determining, by the processor, to apply a differential compression based on a similarity degree between the input data block and the optimal reference data block;
• and
• applying, by the processor, the differential compression to the input data block and the optimal reference data block.