Data compaction utilizing delta encoding

A method for multiple pass data compaction using delta encoding, comprising the steps of:

• receiving a data stream for compaction, the data stream comprising sourceblocks of data;
• for each sourceblock of the data stream:
• calculating a hash of the sourceblock as a sourceblock approximation using the hash function;
• retrieving an approximation codeword for the sourceblock approximation by looking up the calculated hash of the sourceblock;
• inserting the approximation codeword into a primary data stream;
• and
• sending the sourceblock and the approximation codeword to a delta encoder;
• and
• for each sourceblock and approximation codeword received from the primary encoder:
• retrieving a sourceblock hash using the approximation codeword;
• unhashing the sourceblock hash using an inverse of the hash function to obtain a sourceblock approximation;
• calculating a delta value between the sourceblock and the sourceblock approximation;
• retrieving a delta codeword for the delta value;
• and
• inserting the delta codeword into a delta data stream.

Non-transitory, computer-readable storage media having computer-executable instructions embodied thereon that, when executed by one or more processors of a computing system for multiple pass data compaction using delta encoding, cause the computing system to:

• receive a data stream for compaction, the data stream comprising sourceblocks of data;
• for each sourceblock of the data stream:
• calculate a hash of the sourceblock as a sourceblock approximation using the hash function;
• retrieve an approximation codeword for the sourceblock approximation by looking up the calculated hash of the sourceblock;
• insert the approximation codeword into a primary data stream;
• and
• send the sourceblock and the approximation codeword to a delta encoder;
• and
• for each sourceblock and approximation codeword received from the primary encoder:
• retrieve a sourceblock hash using the approximation codeword;
• unhash the sourceblock hash using an inverse of the hash function to obtain a sourceblock approximation;
• calculate a delta value between the sourceblock and the sourceblock approximation;
• retrieve a delta codeword for the delta value;
• and
• insert the delta codeword into a delta data stream.