Systems and methods for transformation of logical data objects for storage

A system for transforming a logical data object for storage in a distributed network, said system comprising:

• a processor;
• and
• memory coupled to the processor, wherein the memory is configured to store computer code that, when executed by the processor, causes the processor to:
• create, in the memory, a transformed logical data object comprising one or more allocated storage sections,
• receive a plurality of data chunks,
• compare a characteristic of the plurality of data chunks to a predefined criterion comprising an estimated or actual time of transformation of the plurality of data chunks, the predefined criterion including an estimated time necessary to transform data which, being transformed, would amount to a transformed section size such that the allocated section size is determined by the estimated time necessary to perform the compression of the raw data chunks which, when compressed, would completely fill the allocated section size,
• if the characteristic matches the predefined criterion:
• transform the plurality of data chunks to create a plurality of transformed data chunks, and
• accumulate the plurality of transformed data chunks in a first storage section,
• if the characteristic does not match the predefined criterion:
• identify the plurality of data chunks as being in a non-transformed form or a partly transformed form, and
• accumulate the non-transformed or partly transformed plurality of data chunks in a second storage section until the second storage section;
• wherein the non-transformed and the partly transformed data chunks are data chunks in which transformation was interrupted.

A computer program product comprising a non-transitory computer readable medium including computer readable program code embodied therein for transforming a logical data object for storage in a distributed network, said computer program product comprising:

• said computer readable program code executed by a processor to perform:
• creating, in the distributed network, a transformed logical data object comprising
• one or more allocated storage sections;
• receiving a plurality of data chunks;
• comparing a characteristic of the plurality of data chunks to a predefined criterion comprising an estimated or actual time of transformation of the plurality of data chunks, the predefined criterion including an estimated time necessary to transform data which, being transformed, would amount to a transformed section size such that the allocated section size is determined by the estimated time necessary to perform the compression of the raw data chunks which, when compressed, would completely fill the allocated section size;
• if the characteristic matches the predefined criterion:
• transforming the plurality of data chunks to create a plurality of transformed data chunks, wherein at least two transformed data chunks include different sizes subsequent to transformation, and
• accumulating the plurality of transformed data chunks in a first storage section;
• if the characteristic does not match the predefined criterion:
• identifying the plurality of data chunks as being in a non-transformed form or a partly transformed form, and
• accumulating the non-transformed or partly transformed plurality of data chunks in a second storage section;
• wherein the non-transformed and the partly transformed data chunks are data chunks in which transformation was interrupted.