Method for reducing primary and backup storage

A non-transitory computer-readable medium configured to store instructions for data management in a data storage system comprising a plurality of data storage units, which, when executed by a processor, cause the processor to perform steps comprising:

• dividing each file in the data storage system into a plurality of blocks having a common size;
• generating a hash value for each block using a common hash algorithm;
• identifying a plurality of similar files in different data storage units, based on a comparison of the hashes for each file;
• copying one or more of the identified similar files, such that similar files are stored in a single data storage unit;
• and
• for one or more of the copied files, generating a link to a new location of the file.

A data management device for a data storage system comprising a plurality of data storage units, the data management device configured to:

• divide each file in the data storage system into a plurality of blocks having a common size;
• generate a hash value for each block using a common hash algorithm;
• identify a plurality of similar files in different data storages, based on a comparison of the hashes for each file;
• copy one or more of the identified similar files, such that similar files are stored in a single data storage;
• and
• for one or more of the copied files, generate a link to a new location of file.