# US 10,372,680 B2 · Systems and methods for transformation of logical data objects for storage

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,372,680 B2
- Document ID: US10372680B2
- Status: Granted
- Kind: B2
- Application number: 15/011,708
- Filed: 2016-02-01
- Published: 2016-05-26
- Granted: 2019-08-06
- Earliest claimed: 2006-05-31
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Jonathan Amit; Nadav Kedem; Chaim Koifman; Avi Zohar
- Web page: https://patentgrep.com/patent/US10372680B2

## CPC

- G06F 16/1744: {using compression, e.g. sparse files}
- G06F 3/0604: {Improving or facilitating administration, e.g. storage management}
- G06F 3/0605: {by facilitating the interaction with a user or administrator}
- G06F 3/0608: {Saving storage space on storage systems}
- G06F 3/0623: {in relation to content}
- G06F 3/0631: {by allocating resources to storage systems}
- G06F 3/0638: {Organizing or formatting or addressing of data}
- G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling}
- G06F 3/0661: {Format or protocol conversion arrangements}
- G06F 3/0665: {at area level, e.g. provisioning of virtual or logical volumes}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G06F 3/0673: {Single storage device}
- G06F 12/023: {Free address space management}
- G06F 12/1408: {by using cryptography (for digital transmission H04L9/00)}
- G06F 16/13: File access structures, e.g. distributed indices (arrangements of input from, or output to, record carriers G06F3/06)
- G06F 16/1727: {Details of free space management performed by the file system (saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064)}
- G06F 16/2365: {Ensuring data consistency and integrity}
- G06F 21/62: Protecting access to data via a platform, e.g. using keys or access control rules
- G06F 21/6209: {to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself}
- G06F 21/78: to assure secure storage of data (address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00)
- H03M 7/30: Compression (speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N); Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- G06F 2212/1044: Space efficiency improvement
- G06F 2212/152: Virtualized environment, e.g. logically partitioned system
- G06F 2212/401: Compressed data
- G06F 2212/402: Encrypted data
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)

## Abstract

Systems and methods for compressing a raw logical data object (201) for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimizatic in and restoring thereof. Compressing the raw logical data object (201) comprises creating in the storage device a compressed logical data object (203) comprising a header (204) and one or more allocated compressed sections with predefined size (205-1-205-2); compressing one or more sequentially obtained chunks of raw data (202-1-202-6) corresponding to the raw logical data object (201) thus giving rise to the compressed data chunks (207-1-207-6); and sequentially accommodating the processed data chunks into: said compressed sections (205-1-205-2) in accordance with an order said chunks received, wherein said compressed sections serve as atomic elements of compression/decompression operations during input/output transactions on the logical data object.

## Claims

### Claim 1

A method for writing data to a storage device in a distributed network, by a processor executing computer code stored in a memory, said method comprising:
generating, in the storage device by the processor, a compressed logical data object comprising a plurality of allocated sections;
receiving, by the processor, chunks of raw data;
comparing, by the processor, a characteristic of the raw data chunks to a predefined criterion comprising an estimated time necessary to compress the raw data chunks which, being compressed, would amount to an allocated section size such that the allocated section size is determined by the estimated time necessary to perform the compression of the raw data chunks which, when compressed, would completely fill the allocated section size;
if the characteristic matches the predefined criterion:
sequentially compressing, by the processor, the raw data chunks to generate compressed data chunks,
accumulating, by the processor, the compressed data chunks in a first allocated section, and
writing, by the processor, said accumulated compressed data chunks into said first allocated section; and
if the characteristic does not match the predefined criterion:
identifying, by the processor, the raw data chunks as being in a non-compressed form or a partly compressed form,
accumulating, by the processor, the non-compressed or partly compressed data chunks in a second allocated section, and
writing, by the processor, said accumulated non-compressed or partly compressed data chunks into said second allocated section.

### Claim 2 (depends on 1)

The method of claim 1, further comprising mapping, by the processor, the data in the compressed logical data object to the data stored in each allocated section.

### Claim 3 (depends on 1)

The method of claim 1, wherein each allocated section comprises a same size.

### Claim 4 (depends on 1)

The method of claim 1, wherein a size of each allocated section is configurable in accordance with a predetermined criterion.

### Claim 5 (depends on 1)

The method of claim 1, wherein compressing the chunks of raw data is performed via an adaptive dictionary, and the data chunks are stored in a same allocated section via a same dictionary.

### Claim 6 (depends on 1)

The method of claim 1, wherein compressing the chunks of raw data is performed via a plurality of adaptive dictionaries, and at least two data chunks are stored in different allocated sections via different dictionaries.

### Claim 7 (depends on 1)

The method of claim 1, wherein each allocated section is associated with a flag indicating a use/re-use condition of a respective physical location of the allocated section, and each data chunk is associated with a same flag as the allocated section storing each data chunk.

### Claim 8

A system for compressing a raw logical data object for storage in a storage device in a distributed network, said system comprising:
a processor; and
memory coupled to said processor, wherein the processor is configured for storing computer code that, when executed by the processor, causes the processor to:
generate, in the storage device, a compressed logical data object comprising a plurality of allocated sections,
receive chunks of raw data,
compare a characteristic of the raw data chunks to a predefined criterion comprising an estimated time necessary to compress the raw data chunks which, being compressed, would amount to an allocated section size such that the allocated section size is determined by the estimated time necessary to perform the compression of the raw data chunks which, when compressed, would completely fill the allocated section size,
if the characteristic matches the predefined criterion:
sequentially compress the raw data chunks to generate compressed data chunks,
accumulate the compressed data chunks in a first allocated section, and
write said accumulated compressed data chunks into said first allocated section, and
if the characteristic does not match the predefined criterion:
identify the raw data chunks as being in a non-compressed form or a partly compressed form,
accumulate the non-compressed or partly compressed data chunks in a second allocated section, and
write said accumulated non-compressed or partly compressed data chunks into said second allocated section.

### Claim 9 (depends on 8)

The system of claim 8, wherein the processor is further configured to map the data in the compressed logical data object to the data stored in each compressed section.

### Claim 10 (depends on 8)

The system of claim 8, wherein each allocated section comprises a same size.

### Claim 11 (depends on 8)

The system of claim 8, wherein a size of each allocated section is configurable in accordance with a predetermined criterion.

### Claim 12 (depends on 8)

The system of claim 8, further comprising an adaptive dictionary, wherein compression is provided via an adaptive dictionary, and the data chunks are stored in a same allocated section via a same dictionary.

### Claim 13 (depends on 8)

The system of claim 8, further comprising a plurality of adaptive dictionaries, wherein compression is provided via an adaptive dictionary, and at least two data chunks are stored in different allocated sections via different dictionaries.

### Claim 14 (depends on 8)

The system of claim 8, wherein each allocated section is associated with a flag indicating a use/re-use condition of a respective physical location of the allocated section, and each data chunk is associated with a same flag as the allocated section storing each data chunk.

### Claim 15

A computer program product comprising a non-transitory computer-useable medium including a computer readable program code embodied therein for compressing a raw logical data object for storage in a storage device in a distributed network, said computer program product comprising:
computer code, executed by a processor, for:
generating, in the storage device, a compressed logical data object comprising a plurality of allocated sections;
receiving chunks of raw data;
comparing a characteristic of the raw data chunks to a predefined criterion comprising an estimated time necessary to compress the raw data chunks which, being compressed, would amount to an allocated section size such that the allocated section size is determined by the estimated time necessary to perform the compression of the raw data chunks which, when compressed, would completely fill the allocated section size;
if the characteristic matches the predefined criterion:
sequentially compressing the raw data chunks to generate compressed data chunks,
accumulating the compressed data chunks in a first allocated section, and
writing said accumulated compressed data chunks into said first allocated section; and
if the characteristic does not match the predefined criterion:
identifying the raw data chunks as being in a non-compressed form or a partly compressed form,
accumulating the non-compressed or partly compressed data chunks in a second allocated section, and
writing said accumulated non-compressed or partly compressed data chunks into said second allocated section.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the computer code maps the data in the compressed logical data object to the data stored in each allocated section.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein each allocated section comprises a same size.

### Claim 18 (depends on 15)

The computer program product of claim 15, wherein a size of each allocated section is configurable in accordance with a predetermined criterion.

### Claim 19 (depends on 15)

The computer program product of claim 15, wherein the compression is provided via an adaptive dictionary, and the data chunks are stored in a same allocated section via a same dictionary.

### Claim 20 (depends on 15)

The computer program product of claim 15, wherein:
the compression is provided via a plurality of adaptive dictionaries, and at least two data chunks are stored in different allocated sections via different dictionaries;
each compressed section is associated with a flag indicating a use/re-use condition of a respective physical location of the allocated section; and
each data chunk is associated with a same flag as the allocated section storing each data chunk.