System and method for file system level compression using compression group descriptors

A non-transitory computer-readable media containing executable program instructions executed by a processor, comprising:

• program instructions that receive write data at a storage system;
• program instructions that compress the write data using a compression technique;
• program instructions that determine whether the compressed data meets a predefined compression threshold, wherein the predefined compression threshold is met when a number of storage blocks used to store the compressed data is less than a predefined number of storage blocks;
• program instructions that generate a compression group comprising a header having a first predefined bit value identifying a first valid physical block within a block address space of the storage system, one or more pointers to the compressed data, and a footer having a second predefined bit value identifying a second valid physical block within the block address space of the storage system, in response to determining that the compressed data meets the predefined compression threshold;
• program instructions that generate the compression group comprising one or more pointers to the write data without compressing the write data in response to determining that the compressed data does not meet the predefined threshold;
• program instructions that write the compression group to storage devices associated with the storage system;
• program instructions that receive a request, associated with data, at the storage system from a client;
• program instructions that determine that the read request is directed to the compression group of a plurality of compression groups stored on the storage devices;
• program instructions that determine if the first predefined bit value is in the header of the compression group;
• program instructions that determine that the compression group contains compressed data, program instructions that decompress the compressed data in the compression group, and program instructions that return the data to the client, in response to determining that the first predefined bit value is in the header of the compression group;
• and
• program instructions that determine that the compression group does not contain the compressed data and program instructions that return the data from the compression group to the client without performing a decompression operation, in response to determining that the first predefined bit value is not in the header of the compression group.

A system for writing data, the system comprising:

• a storage operating system executing on a storage system, the storage operating system comprising a file system configured to write data to one or more storage devices;
• a compression module operatively interconnected with the file system, the compression module configured to:
• compress a set of write data, determine whether the compressed set of write data meets a predefined compression threshold wherein the predefined compression threshold is met when a number of storage blocks used to store compressed data is less than a predefined number of storage blocks,
• generate a compression group that comprises a header having a value indicating that the compression group comprises the compressed set of write data, one or more pointers to the compressed set of write data, and a footer, in response to determining that the compressed set of write data meets the predefined compression threshold, and
• generate the compression group that comprises one or more pointers to the set of write data without compressing the set of write data in response to determining that the compressed set of write data does not meet the predefined compression threshold;
• one or more processors operatively interconnected with the file system, the one or more processors configured to:
• write the compression group to the storage device,
• receive a read request, associated with data, at the storage system from a client;
• determine that the read request is directed to the compression group,
• determine if the value is in the header of the compression group,
• determine that the compression group contains the compressed data, decompress the compressed data in the compression group, and return the data, obtained from decompressing the compressed data, to the client, in response to determining that the value is in the header of the compression group, and
• determine that the compression group does not contain the compressed data, return the data from the compression group to the client without performing a decompression operation, in response to determining that the value is not in the header of the compression group.