Systems and methods for version chain clustering

A system for storing data, comprising:

• at least one processor;
• and
• at least one machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
• receiving a data stream, wherein the data stream includes a plurality of transactions, each transaction in the plurality of transactions includes a plurality of versions of at least one data file, each transaction in the plurality of transactions represents a logical unit of the data stream;
• determining whether at least one first version in the plurality of versions within a first transaction in the plurality of transactions is similar to at least one second version in the plurality of versions within the first transaction;
• grouping, based on the determination of similarity of versions within the first transaction, at least a portion of the plurality of versions within the first transaction into a cluster;
• selecting, based on grouping, a representative of the first transaction to represent all versions within the first transaction for storing, the representative being a version in the plurality of versions in the first transaction, wherein the grouped versions within the first transaction not selected as the representative are determined to be dependent on the selected representative, each version within the first transaction not selected as the representative is recomputed to represent a content difference between the selected representative and that version, wherein versions in the first transaction not selected as the representative are independent of one another;
• identifying, based on the determining, the grouping and the selecting, a representative for each transaction in the plurality of transactions;
• and
• storing each identified representative, wherein a plurality of identified representatives is configured to form a chain representing the received data stream.

A computer program product comprising non-transitory machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

• receiving a data stream, wherein the data stream includes a plurality of transactions, each transaction in the plurality of transactions includes a plurality of versions of at least one data file, each transaction in the plurality of transactions represents a logical unit of the data stream;
• determining whether at least one first version in the plurality of versions within a first transaction in the plurality of transactions is similar to at least one second version in the plurality of versions within the first transaction;
• grouping, based on the determination of similarity of versions within the first transaction, at least a portion of the plurality of versions within the first transaction into a cluster;
• selecting, based on grouping, a representative of the first transaction to represent all versions within the first transaction for storing, the representative being a version in the plurality of versions in the first transaction, wherein the grouped versions within the first transaction not selected as the representative are determined to be dependent on the selected representative, each version within the first transaction not selected as the representative is recomputed to represent a content difference between the selected representative and that version, wherein versions in the first transaction not selected as the representative are independent of one another;
• identifying, based on the determining, the grouping and the selecting, a representative for each transaction in the plurality of transactions;
• and
• storing each identified representative, wherein a plurality of identified representatives is configured to form a chain representing the received data stream.