# US 11,907,531 B2 · Optimizing storage-related costs with compression in a multi-tiered storage device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,531 B2 - Document ID: US11907531B2 - Status: Granted - Kind: B2 - Application number: 17/851,443 - Filed: 2022-06-28 - Published: 2023-12-28 - Granted: 2024-02-20 - Earliest claimed: 2022-06-28 - Assignees: Adobe Inc. - Applicants: Adobe Inc. - Inventors: Raunak Shah; Koyel Mukherjee; Khushi; Kavya Barnwal; Karanpreet Singh; Harsh Kesarwani; Ayush Chauhan - Web page: https://patentgrep.com/patent/US11907531B2 ## CPC - G06F 3/0604: {Improving or facilitating administration, e.g. storage management} - G06F 3/0685: {Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays} - G06F 3/0608: {Saving storage space on storage systems} - G06F 3/0644: {Management of space entities, e.g. partitions, extents, pools} - G06F 3/0649: {Lifecycle management} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - H03M 7/6064: {Selection of Compressor} - H03M 7/6082: {Selection strategies} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C) ## Abstract Some techniques described herein relate to determining how to optimally store datasets in a multi-tiered storage device with compression. In one example, a method includes assigning, to a data partition of a dataset, a priority based on access patterns of the data partition. Compression data is accessed describing results of compressing a data sample associated with the data partition using multiple compression schemes. Based both on the priority of the data partition and the compression data, a storage tier is determined for storing the data partition in the multi-tiered storage device. Further, based both on the priority of the data partition and the compression data, a compression scheme is determined for compressing the data partition for storage in the multi-tiered storage device. The data partition is compressed using the compression scheme to produce a compressed data partition, and the compressed data partition is stored in the storage tier. ## Claims ### Claim 1 A method comprising: assigning, to a data partition of a dataset, a priority based on access patterns of the data partition; accessing compression data describing results of compressing a data sample associated with the data partition using multiple compression schemes; determining, based both on the priority of the data partition and on the compression data, a storage tier for storing the data partition in a multi-tiered storage device; determining, based both on the priority of the data partition and on the compression data, a compression scheme for compressing the data partition for storage in the multi-tiered storage device; and compressing the data partition using the compression scheme to produce a compressed data partition and storing the compressed data partition in the storage tier. ### Claim 2 (depends on 1) The method of claim 1, further comprising: assigning, to a second data partition of the dataset, a second priority based on access patterns of the second data partition; accessing second compression data describing results of compressing a data sample associated with the second data partition using the multiple compression schemes; determining, based both on the second priority of the second data partition and on the compression data, a second storage tier for storing the second data partition in the multi-tiered storage device, wherein the second storage tier differs from the storage tier; determining, based both on the second priority of the second data partition and on the compression data, a second compression scheme for compressing the second data partition for storage in the multi-tiered storage device, wherein the second compression scheme differs from the compression scheme; and compressing the second data partition using the second compression scheme to produce a compressed second data partition and storing the compressed second data partition in the second storage tier. ### Claim 3 (depends on 1) The method of claim 1, further comprising: assigning, to a second data partition, a second priority based on access patterns of the second data partition; accessing second compression data describing results of compressing a second data sample associated with the second data partition using the multiple compression schemes; identifying, based both on the second priority of the second data partition and on the compression data, the storage tier for storing the second data partition in the multi-tiered storage device; determining, based both on the second priority of the second data partition and on the compression data, a second compression scheme for compressing the second data partition for storage in the multi-tiered storage device, wherein the second compression scheme differs from the compression scheme; and compressing the second data partition using the second compression scheme to produce a compressed second data partition and storing the compressed second data partition in the storage tier with the compressed data partition. ### Claim 4 (depends on 1) The method of claim 1, further comprising: computing priorities of data in the dataset based on access logs describing accesses to the dataset; dividing the dataset into data partitions based on the priorities; and determining the data partition from the data partitions. ### Claim 5 (depends on 4) The method of claim 4, wherein determining the data partition from the data partitions comprises: identifying a first data partition of the data partitions, wherein the first data partition is associated with a first priority; identifying a second data partition of the data partitions, wherein the second data partition is associated with a second priority; determining that the first priority of the first data partition and the second priority of the second data partition are within a threshold range of each other; and based on the first priority and the second priority being within a threshold range of each other, merging the first data partition and the second data partition to produce the data partition, wherein the data partition is smaller than a sum of a size the first data partition and a size of the second data partition. ### Claim 6 (depends on 1) The method of claim 1, wherein accessing the compression data describing results of compressing the data sample associated with the data partition using multiple compression schemes comprises determining the compression data, and wherein determining the compression data comprises: compressing the data sample using the compression scheme selected from the multiple compression schemes; providing, in the compression data, an indication of a compression ratio and a decompression time for the data sample using the compression scheme; compressing the data sample using a second compression scheme selected from the multiple compression schemes; and providing, in the compression data, an indication of a second compression ratio and a second decompression time for the data sample using the second compression scheme. ### Claim 7 (depends on 6) The method of claim 6, wherein determining the storage tier and determining the compression scheme comprise optimizing a cost function based on the compression data. ### Claim 8 (depends on 1) The method of claim 1, wherein determining the storage tier and determining the compression scheme comprise optimizing a cost function based both on the priority of the data partition and on the compression data. ### Claim 9 (depends on 8) The method of claim 8, wherein the cost function is further based on an overall storage cost of storing the dataset in the multi-tiered storage device. ### Claim 10 A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising: assigning, to a data partition of a dataset, a priority based on access patterns of the data partition; accessing compression data describing results of compressing a data sample associated with the data partition using multiple compression schemes; determining, based both on the priority of the data partition and on the compression data, a storage tier for storing the data partition in a multi-tiered storage device; determining, based both on the priority of the data partition and on the compression data, a compression scheme for compressing the data partition for storage in the multi-tiered storage device; and compressing the data partition using the compression scheme to produce a compressed data partition and storing the compressed data partition in the storage tier. ### Claim 11 (depends on 10) The non-transitory computer-readable medium of claim 10, the operations further comprising: computing priorities of data in the dataset based on access logs describing accesses to the dataset; dividing the dataset into data partitions based on the priorities; and determining the data partition from the data partitions. ### Claim 12 (depends on 11) The non-transitory computer-readable medium of claim 11, wherein determining the data partition from the data partitions comprises: identifying a first data partition of the data partitions, wherein the first data partition is associated with a first priority; identifying a second data partition of the data partitions, wherein the second data partition is associated with a second priority; determining that the first priority of the first data partition and the second priority of the second data partition are within a threshold range of each other; and based on the first priority and the second priority being within a threshold range of each other, merging the first data partition and the second data partition to produce the data partition, wherein the data partition is smaller than a sum of a size the first data partition and a size of the second data partition. ### Claim 13 (depends on 10) The non-transitory computer-readable medium of claim 10, wherein accessing the compression data describing results of compressing the data sample associated with the data partition using multiple compression schemes comprises determining the compression data, and wherein determining the compression data comprises: compressing the data sample using the compression scheme selected from the multiple compression schemes; providing, in the compression data, an indication of a compression ratio and a decompression time for the data sample using the compression scheme; compressing the data sample using a second compression scheme selected from the multiple compression schemes; and providing, in the compression data, an indication of a second compression ratio and a second decompression time for the data sample using the second compression scheme. ### Claim 14 (depends on 10) The non-transitory computer-readable medium of claim 10, wherein determining the storage tier and determining the compression scheme comprise optimizing a cost function based both on the priority of the data partition and on the compression data. ### Claim 15 (depends on 14) The non-transitory computer-readable medium of claim 14, wherein the cost function is further based on an overall storage cost of storing the dataset in the multi-tiered storage device. ### Claim 16 A system comprising: a memory component storing a dataset comprising data partitions; a data partitioner module comprising program code for assigning, to a data partition of the data partitions, a priority based on access patterns of the data partition; an optimization engine comprising program code for: accessing compression data describing results of compressing a data sample associated with the data partition using multiple compression schemes; determining, based both on the priority of the data partition and on the compression data, a storage tier for storing the data partition in a multi-tiered storage device; and determining, based both on the priority of the data partition and on the compression data, a compression scheme for compressing the data partition for storage in the multi-tiered storage device; a storage manager module comprising program code for compressing the data partition using the compression scheme to produce a compressed data partition and storing the compressed data partition in the storage tier; and a processing device coupled to the memory component and configured to execute the program code of the data partitioner module, the optimization engine, and the storage manager module. ### Claim 17 (depends on 16) The system of claim 16, wherein the data partitioner module is further configured to perform operations comprising: computing priorities of data in the dataset based on access logs describing accesses to the dataset; and dividing the dataset into the data partitions based on the priorities. ### Claim 18 (depends on 17) The system of claim 17, wherein the data partitioner module is further configured to perform operations comprising: identifying a first data partition of the data partitions, wherein the first data partition is associated with a first priority; identifying a second data partition of the data partitions, wherein the second data partition is associated with a second priority; determining that the first priority of the first data partition and the second priority of the second data partition are within a threshold range of each other; and based on the first priority and the second priority being within a threshold range of each other, merging the first data partition and the second data partition to produce the data partition, wherein the data partition is smaller than a sum of a size the first data partition and a size of the second data partition. ### Claim 19 (depends on 16) The system of claim 16, further comprising a compression predictor module comprising program code for determining the compression data, wherein to determine the compression data the compression predictor module performs operations comprising: compressing the data sample using the compression scheme selected from the multiple compression schemes; providing, in the compression data, an indication of a compression ratio and a decompression time for the data sample using the compression scheme; compressing the data sample using a second compression scheme selected from the multiple compression schemes; and providing, in the compression data, an indication of a second compression ratio and a second decompression time for the data sample using the second compression scheme. ### Claim 20 (depends on 16) The system of claim 16, wherein the optimization engine is further configured to determine the storage tier and determine the compression scheme by optimizing a cost function based both on the priority of the data partition and on the compression data.