Auto-tuned write-optimized key-value store
AssigneeVMware, Inc.
InventorsWenguang Wang, Abhishek Gupta, Richard P Spillane, Kapil Chowksey, Robert T Johnson
Life of this patent
Estimating the term…
Abstract
A Bε-tree associated with a file system on a storage volume includes a hierarchy of nodes. Each node includes a buffer portion to store key-value pairs as messages in the buffer. Each node can be characterized by having a maximum allowable size that is periodically updated at run time. The buffers in the nodes of the Bε-tree are therefore characterized by having a maximum allowed size that can vary over time.
Claims20
A method for a file system on a storage volume, the method comprising:
- receiving modify operations and query operations on a tree data structure of the file system, the tree data structure comprising a plurality of nodes organized in a hierarchy, each node having a buffer comprising one or more disk blocks allocated from the storage volume for storing data, the size of each node varying depending on the amount of data stored in its buffer;
- periodically updating a value of an expected node size parameter (Bexp);
- processing a first modify operation by inserting write data associated with the first modify operation to the buffer in a root node of the tree data structure;
- triggering a cascade of data movement to one or more children nodes descending from the root node by flushing data in the buffer of the root node to a child node of the root node when an amount of data stored in the buffer of the root node exceeds a threshold;
- and
- enforcing the current value of Bexp on any children nodes visited during the cascade of data movement including flushing at least some data in the buffer of a visited child node when the size of the visited child node exceeds the current value of Bexp, wherein the current value of Bexp changes as the value of Bexp is periodically updated.
A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computer device, cause the computer device to:
- receive s and query operations on a tree data structure of the file system, the tree data structure comprising a plurality of nodes organized in a hierarchy, each node having a buffer comprising one or more disk blocks allocated from the storage volume for storing data, the size of each node varying depending on the amount of data stored in its buffer;
- periodically update a value of an expected node size parameter (Bexp);
- process by inserting write data associated with the to the buffer in a root node of the tree data structure;
- trigger a cascade of data movement to one or more children nodes descending from the root node by flushing data in the buffer of the root node to a child node of the root node when an amount of data stored in the buffer of the root node exceeds a threshold;
- and
- enforce the current value of Bexp on any children nodes visited during the cascade of data movement including flushing at least some data in the buffer of a visited child node when the size of the visited child node exceeds the current value of Bexp, wherein the current value of Bexp changes as the value of Bexp is periodically updated.
An apparatus comprising:
- one or more computer processors;
- and
- a computer-readable storage medium comprising instructions for controlling the one or more computer processors to be operable to:
- receive s and query operations on a tree data structure of the file system, the tree data structure comprising a plurality of nodes organized in a hierarchy, each node having a buffer comprising one or more disk blocks allocated from the storage volume for storing data, the size of each node varying depending on the amount of data stored in its buffer;
- periodically update a value of an expected node size parameter (Bexp);
- process a first by inserting write data associated with the to the buffer in a root node of the tree data structure;
- trigger a cascade of data movement to one or more children nodes descending from the root node by flushing data in the buffer of the root node to a child node of the root node when an amount of data stored in the buffer of the root node exceeds a threshold;
- and
- enforce the current value of Bexp on any children nodes visited during the cascade of data movement including flushing at least some data in the buffer of a visited child node when the size of the visited child node exceeds the current value of Bexp, wherein the current value of Bexp changes as the value of Bexp is periodically updated.
Dependencies are inferred from the claim text.