Compression techniques for distributed data
AssigneeIntel Corporation
InventorsJawad B. KHAN, Sanjeev N. TRIKA
Life of this patent
Estimating the term…
Abstract
In one example, uncompressed data is compressed and divided into chunks. Each chunk of the compressed data stream is combined with state information to enable each chunk to be independently decompressed. Each of the compressed chunks is then stored on a different storage device along with its associated state information. A compute operation can then be offloaded to the device or node where each chunk is stored. Each chunk can be independently decompressed for execution of the offloaded operation without transferring all chunks to a central location for decompression and performance of the operation.
Claims20
An article of manufacture comprising a computer readable storage medium having content stored thereon which when accessed causes processing circuitry to execute operations to perform a method comprising:
- receiving, at a storage device, a request to perform an operation on a chunk of compressed data;
- decompressing the chunk of compressed data with its associated state information independently from other chunks of the compressed data;
- performing the operation on the decompressed data;
- and
- providing a result from the operation.
An article of manufacture comprising a computer readable storage medium having content stored thereon which when accessed causes processing circuitry to execute operations to perform a method comprising:
- receiving uncompressed data;
- compressing the uncompressed data to generate compressed data;
- dividing the compressed data into chunks;
- generating state information for each chunk of the compressed data, each chunk independently de-compressible with its associated state information;
- and
- storing the compressed data on a plurality of storage devices, each chunk of the compressed data to be stored on a same storage device as its associated state information.
A storage node comprising:
- input/output (I/O) interface logic to:
- receive a request to perform an operation to access compressed data, a chunk of the compressed data and its associated state information to be stored on the storage node;
- and
- logic to:
- decompress the chunk at the storage node with its associated state information independently from other chunks of the compressed data,
- perform the operation on the decompressed data, and
- provide a result from the operation.
Dependencies are inferred from the claim text.