# US 2014/0195498 A1 · Real-time reduction of cpu overhead for data compression > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0195498 A1 - Document ID: US20140195498A1 - Status: Published application - Kind: A1 - Application number: 13/738,300 - Filed: 2013-01-10 - Published: 2014-07-10 - Earliest claimed: 2013-01-10 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Ron ASHER; Danny HARNIK; Oded MARGALIT; Kat I. RONEN; Dmitry SOTNIKOV - Web page: https://patentgrep.com/patent/US20140195498A1 ## CPC - H03M 7/4037: {Prefix coding} - H03M 7/607: {Selection between different types of compressors} - H03M 7/6088: {according to the data type} - H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C) ## Abstract Real-time reduction of CPU overhead for data compression is performed by a processor device in a computing environment. Non-compressing heuristics are applied on a randomly selected data sample from data sequences for determining whether to compress the data sequences. A compression potential is calculated based on the non-compressing heuristics. The compression potential is compared to a threshold value. The data sequences are either compressed if the compress threshold is matched, compressed using Huffman coding if Huffman coding threshold is matched, or stored without compression. ## Claims ### Claim 1 A method for real-time reduction of CPU overhead for data compression by a processor device in a computing environment, the method comprising: applying non-compressing heuristics on a randomly selected data sample from data sequences for determining whether to compress the data sequences by calculating a compression potential based on the non-compressing heuristics, wherein the compression potential is compared to a threshold value; and performing one of: compressing the data sequences if the compress threshold is matched, compressing the data sequences using Huffman coding if Huffman coding threshold is matched, and storing the data sequences without compression. ### Claim 2 (depends on 1) The method of claim 1, further including: using at least one of the non-compressing heuristics, selecting a randomly selected data sample, and computing core characters that compose a predefined percentage of bytes in the randomly selected data sample, an entropy of the data, a relation between appearances of character pairs and a random distribution of the character pairs, and the entropy of the character pairs for determining whether to compress the data sequences. ### Claim 3 (depends on 1) The method of claim 1, further including calculating the non-compressing heuristics one after another for developing a heuristic score for making a decision for determining whether to compress the data sequences, wherein the CPU speed is optimized. ### Claim 4 (depends on 3) The method of claim 3, further including calculating the compression potential based on the heuristic score of the non-compressing heuristics. ### Claim 5 (depends on 1) The method of claim 1, further including: estimating a relation between appearances of character pairs and a random distribution of the character pairs by: comparing a number of pairs from core characters in the randomly selected data sample against the number of pairs from the core characters expected to appear in a random distribution in the randomly selected data sample, and calculating a L2 norm distance between a first vector of distributions representing an observed distribution of the number of pairs from the core characters in the randomly selected data sample and a second vector of distributions representing an expected distribution of single values assuming there is no correlation between subsequent pairs from the core characters. ### Claim 6 (depends on 1) The method of claim 1, further including performing one of turning off and turning on the calculating the compression potential, and determining whether to compress the data sequences according to a predefined setting. ### Claim 7 (depends on 1) The method of claim 1, wherein the predefined setting, further includes at least one of: continuously applying the non-compressing heuristics and providing an indication as to whether to compress the data sequences, applying the non-compressing heuristics on demand when a compression ratio is above the predetermined threshold for a predefined number of the data sequences, applying the non-compressing heuristics according to a size of the buffer, and applying a prefix compression estimation and deciding whether to compress the data sequences based on a prefix compression ratio when the prefix compression ratio of the data sequences is below a threshold. ### Claim 8 (depends on 1) The method of claim 1, further including applying the non-compressing heuristics to a relation between character pairs of core characters and random distributions of the character pairs of the core characters from the randomly selected data sample from data sequences for determining whether to compress the data sequences by calculating the compression potential based on the non-compressing heuristics. ### Claim 9 A system for real-time reduction of CPU overhead for data compression in a computing environment, the system comprising: a processor device operable in the computing storage environment, wherein the processor device: applies non-compressing heuristics on a randomly selected data sample from data sequences for determining whether to compress the data sequences by calculating a compression potential based on the non-compressing heuristics, wherein the compression potential is compared to a threshold value, and performs one of: compressing the data sequences if the compress threshold is matched, compressing the data sequences using Huffman coding if Huffman coding threshold is matched, and storing the data sequences without compression. ### Claim 10 (depends on 9) The system of claim 9, wherein the processor device: uses at least one of the non-compressing heuristics, selects a randomly selected data sample, and computes core characters that compose a predefined percentage of bytes in the randomly selected data sample, an entropy of the data, a relation between appearances of character pairs and a random distribution of the character pairs, and the entropy of the character pairs for determining whether to compress the data sequences. ### Claim 11 (depends on 10) The system of claim 10, wherein the processor device calculates the non-compressing heuristics one after another for developing a heuristic score for making a decision for determining whether to compress the data sequences, wherein the CPU speed is optimized. ### Claim 12 (depends on 11) The system of claim 11, wherein the processor device calculates the compression potential based on the heuristic score of the non-compressing heuristics. ### Claim 13 (depends on 10) The system of claim 10, wherein the processor device: estimates a relation between appearances of character pairs and a random distribution of the character pairs by: comparing a number of pairs from core characters in the randomly selected data sample against the number of pairs from the core characters expected to appear in a random distribution in the randomly selected data sample, and calculating a L2 norm distance between a first vector of distributions representing an observed distribution of the number of pairs from the core characters in the randomly selected data sample and a second vector of distributions representing an expected distribution of single values assuming there is no correlation between subsequent pairs from the core characters. ### Claim 14 (depends on 10) The system of claim 10, wherein the processor device performs one of turning off and turning on the calculating the compression potential, and determining whether to compress the data sequences according to a predefined setting. ### Claim 15 (depends on 10) The system of claim 10, wherein the processor device performs at least one of: continuously applying the non-compressing heuristics and providing an indication as to whether to compress the data sequences, applying the non-compressing heuristics on demand when a compression ratio is above the predetermined threshold for a predefined number of the data sequences, applying the non-compressing heuristics according to a size of the buffer, and applying a prefix compression estimation and deciding whether to compress the data sequences based on a prefix compression ratio when the prefix compression ratio of the data sequences is below a threshold. ### Claim 16 (depends on 10) The system of claim 10, wherein the processor device applies the non-compressing heuristics to a relation between character pairs of core characters and random distributions of the character pairs of the core characters from the randomly selected data sample from data sequences for determining whether to compress the data sequences by calculating the compression potential based on the non-compressing heuristics. ### Claim 17 A computer program product for real-time reduction of CPU overhead for data compression by a processor device by a processor device, the computer program product comprising a computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising: a first executable portion that applies non-compressing heuristics on a randomly selected data sample from data sequences for determining whether to compress the data sequences by calculating a compression potential based on the non-compressing heuristics, wherein the compression potential is compared to a threshold value, and a second executable portion that performs one of: compressing the data sequences if the compress threshold is matched, compressing the data sequences using Huffman coding if Huffman coding threshold is matched, and storing the data sequences without compression. ### Claim 18 (depends on 17) The computer program product of claim 17, further including a third executable portion that: uses at least one of the non-compressing heuristics, selects a randomly selected data sample, and computes core characters that compose a predefined percentage of bytes in the randomly selected data sample, an entropy of the data, a relation between appearances of character pairs and a random distribution of the character pairs, and the entropy of the character pairs for determining whether to compress the data sequences. ### Claim 19 (depends on 17) The computer program product of claim 17, further including a third executable portion that calculates the non-compressing heuristics one after another for developing a heuristic score for making a decision for determining whether to compress the data sequences, wherein the CPU speed is optimized. ### Claim 20 (depends on 19) The computer program product of claim 19, further including a fourth executable portion that calculates the compression potential based on the heuristic score of the non-compressing heuristics. ### Claim 21 (depends on 17) The computer program product of claim 17, further including a third executable portion that: estimates a relation between appearances of character pairs and a random distribution of the character pairs by: comparing a number of pairs from core characters in the randomly selected data sample against the number of pairs from the core characters expected to appear in a random distribution in the randomly selected data sample, and calculating a L2 norm distance between a first vector of distributions representing an observed distribution of the number of pairs from the core characters in the randomly selected data sample and a second vector of distributions representing an expected distribution of single values assuming there is no correlation between subsequent pairs from the core characters. ### Claim 22 (depends on 17) The computer program product of claim 17, further including a third executable portion that performs one of turning off and turning on the calculating the compression potential, and determining whether to compress the data sequences according to a predefined setting. ### Claim 23 (depends on 17) The computer program product of claim 17, further including a third executable portion that performs at least one of: continuously applying the non-compressing heuristics and providing an indication as to whether to compress the data sequences, applying the non-compressing heuristics on demand when a compression ratio is above the predetermined threshold for a predefined number of the data sequences, applying the non-compressing heuristics according to a size of the buffer, and applying a prefix compression estimation and deciding whether to compress the data sequences based on a prefix compression ratio when the prefix compression ratio of the data sequences is below a threshold. ### Claim 24 (depends on 17) The computer program product of claim 17, further including a third executable portion that applies the non-compressing heuristics to a relation between character pairs of core characters and random distributions of the character pairs of the core characters from the randomly selected data sample from data sequences for determining whether to compress the data sequences by calculating the compression potential based on the non-compressing heuristics.