Commonality factoring for removable media

A method for storing data with a plurality of removable media, comprising steps of:

• breaking an original data stream into chunks;
• calculating an identifier for each chunk;
• storing the identifiers;
• determining, based on the identifiers, whether each chunk is unique;
• and
• storing on the plurality of removable media, at least two of a following:
• a stream of the unique chunks,
• a stream of descriptors describing each of the unique chunks, and
• a stream of references indicating a sequence of the unique chunks usable to reconstruct the original data stream, wherein:
• the stream of references stored in each medium includes a medium identifier, and
• the medium identifier allows correlation of the references to the chunks on the different removable media.

A method for storing data on a removable storage unit, comprising:

• breaking original data into chunks;
• calculating an identifier for each chunk;
• storing the identifiers in a low latency memory;
• determining, based on the identifiers, whether each chunk is unique;
• and
• storing on the storage unit at least two of a following:
• a stream of the unique chunks,
• a stream of descriptors describing each of the unique chunks, or
• a stream of references indicating a sequence of the unique chunks used to reconstruct the original data, wherein the at least two streams of the unique chunks, the descriptors or the references are each stored as a separate file.