System and methods for secure deduplication of compacted data

A system for secure deduplication of compacted data, comprising:

• at least one reference codebook comprising key-value pairs of data;
• a library manager comprising at least a processor, a memory, and a plurality of programming instructions stored in the memory and operable on the processor of a computing device, wherein the plurality of programming instructions, when operating on the processor, cause the processor to:
• receive a plurality of deconstructed sourceblocks from a data deconstruction engine;
• perform data deduplication by comparing each of the plurality of deconstructed sourceblocks with sourceblocks already contained in the reference codebook;
• return the reference code to the data deconstruction engine, when the sourceblock received is a duplicate of an existing sourceblock in the reference codebook;
• and
• for each received deconstructed sourceblock that is not present in the codebook:
• create a new, unique reference code for the respective deconstructed sourceblock;
• store both the respective deconstructed sourceblock and the associated reference code in the reference codebook as a key-value pair;
• and
• return the new reference code to the data deconstruction engine.