Data reduction techniques in a flash-based key/value cluster storage
AssigneeEMC IP Holding Company LLC
InventorsDavid Meiri, Anton Kucherov, Vladimir Shveidel
Life of this patent
Estimating the term…
Abstract
In one aspect, a method includes splitting empty RAID stripes into sub-stripes and storing pages into the sub-stripes based on a compressibility score. In another aspect, a method includes reading pages from 1-stripes, storing compressed data in a temporary location, reading multiple stripes, determining compressibility score for each stripe and filling stripes based on the compressibility score. In a further aspect, a method includes scanning a dirty queue in a system cache, compressing pages ready for destaging, combining compressed pages in to one aggregated page, writing one aggregated page to one stripe and storing pages with same compressibility score in a stripe.
Claims19
A method, comprising:
- reading pages from 1-stripes, wherein 1-stripes are not compressed;
- storing compressed data in a temporary location;
- reading j 1-stripes, where j is greater than or equal to one;
- determining compressibility score, S, for each 1-stripe of j stripes;
- and
- filling S-stripes based on the compressibility score by writing compressed pages to S-stripes corresponding to the compressibility score, S, comprising:
- filling 8-stripes before 4-stripes if 8-stripes exist by locating 8N pages with a score greater than or equal to 8;
- filling 4-stripes before 2-stripes if 4-stripes exits by locating 4N pages with a score greater than or equal to 4;
- filling 2-stripes before 1-stripes if 2-stripes exist by locating 2N pages with a score greater than or equal to 2,
- wherein N>1 and N equals the number of disks minus K, and K is the number of parity disks where K>1.
An apparatus, comprising:
- electronic hardware circuitry configured to:
- read pages from 1-stripes, wherein 1-stripes are not compressed;
- store compressed data in a temporary location;
- read j 1-stripes, where j is greater than or equal to one;
- determine compressibility score, S, for each 1-stripe of j stripes;
- and
- fill S-stripes based on the compressibility score by writing compressed pages to S-stripes corresponding to the compressibility score, S, comprising:
- fill 8-stripes before 4-stripes if 8-stripes exist by locating 8N pages with a score greater than or equal to 8;
- fill 4-stripes before 2-stripes if 4-stripes exits by locating 4N pages with a score greater than or equal to 4;
- fill 2-stripes before 1-stripes if 2-stripes exist by locating 2N pages with a score greater than or equal to 2,
- wherein N>1 and N equals the number of disks minus K, and K is the number of parity disks where K>1.
An article comprising:
- a non-transitory computer-readable medium that stores computer-executable instructions, the instructions causing a machine to:
- read pages from 1-stripes, wherein 1-stripes are not compressed;
- store compressed data in a temporary location;
- read j 1-stripes, where j is greater than or equal to one;
- determine compressibility score, S, for each 1-stripe of j stripes;
- and
- fill S-stripes based on the compressibility score by writing compressed pages to S-stripes corresponding to the compressibility score, S, comprising:
- fill 8-stripes before 4-stripes if 8-stripes exist by locating 8N pages with a score greater than or equal to 8;
- fill 4-stripes before 2-stripes if 4-stripes exits by locating 4N pages with a score greater than or equal to 4;
- fill 2-stripes before 1-stripes if 2-stripes exist by locating 2N pages with a score greater than or equal to 2,
- wherein N>1 and N equals the number of disks minus K, and K is the number of parity disks where K>1.
Dependencies are inferred from the claim text.