Systems and methods for compressing data for distributed ledgers

A method for compressing data for use in a distributed ledger, the method comprising:

• receiving, by a control circuit executing a data compression module, a data set via a network interface;
• transforming and storing, by the control circuit, the data set into a file format of rows and columns in which data of the data set are stored in a columnar fashion;
• applying, by the control circuit, a first compression step to compress each row and write each compressed row into the file format;
• applying, by the control circuit, a second compression step to further compress data after the compressed rows have been written into the file format;
• and
• encoding, by the control circuit, the compressed data and transmitting the encoded data to a first node of a distributed ledger;
• wherein the first compression step and the second compression step apply different compression algorithms, the first compression step applying row-level compression and the second compression step applying file-level compression.