Reading compressed data directly into an in-memory store
AssigneeMicrosoft Technology Licensing, LLC
InventorsKrystian Sakowski, Marius Dumitru, Amir Netz, Cristian Petculescu
Life of this patent
Estimating the term…
Abstract
Example solutions read compressed data from a file in persistent storage, directly into an in-memory store without decompression. Some examples transcode the compression scheme used by the file into the compression scheme used by the in-memory store whereas, in other examples, the in-memory store is compatible with the compression scheme used by the file. In some examples that use transcoding, radix clustering is used to speed up compression dictionary transcoding. The radix clustering minimizes cache misses, thereby increasing the efficiency of memory access. These approaches significantly improve cold start times when responding to queries.
Claims20
A radix clustering system comprising:
- a processor;
- and
- a computer-readable medium storing instructions that are operative upon execution by the processor to:
- read compressed data from a file, the compressed data having a first storage compression scheme with a first storage compression dictionary and a second storage compression dictionary;
- hash entries in the first storage compression dictionary and the second storage compression dictionary;
- sort the hashed entries by their associated hash values;
- update a hash table with the hashed entries;
- and
- based on at least the updated hash table, transcode the first and second storage compression dictionaries into an in-memory compression dictionary.
A computer-implemented method comprising:
- reading compressed data from a file, the compressed data having a first storage compression scheme with a first storage compression dictionary and a second storage compression dictionary;
- hashing entries in the first storage compression dictionary and the second storage compression dictionary;
- sorting the hashed entries by their associated hash values;
- updating a hash table with the hashed entries;
- and
- based on at least the updated hash table, transcoding the first and second storage compression dictionaries into an in-memory compression dictionary.
A computer storage device having computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising:
- reading compressed data from a file, the compressed data having a first storage compression scheme with a first storage compression dictionary and a second storage compression dictionary;
- hashing entries in the first storage compression dictionary and the second storage compression dictionary;
- sorting the hashed entries by their associated hash values;
- updating a hash table with the hashed entries;
- and
- based on at least the updated hash table, transcoding the first and second storage compression dictionaries into an in-memory compression dictionary.
Dependencies are inferred from the claim text.