Efficient append-only file system for high-performance key-value stores
OwnerSheng Qiu, Hao Wang +7inventors
InventorsSheng Qiu, Hao Wang, Zhengyu Yang, Yizheng Jiao, Jianyang Hu, Yang Liu, Jiaxin Ou, Huan Sun, Yinlin Zhang
Life of this patent
Estimating the term…
Abstract
A file system particular for use with key-value stores is provided. The file system can operate in a user space instead of a kernel space. The file system can be an append-only file system. The file system can support use of solid state drives (SSDs) for storage, including zoned SSDs. The file system can include a file manager, a metadata manager, a task scheduler, a space allocator, and a collaborator for collaborating with a key-value store.
Claims18
A system comprising:
- at least one processor;
- and
- at least one computer-readable medium containing instructions which, when executed, cause the at least one processor to perform operations comprising:
- maintaining separation of data and logs within a file system, wherein a disk layout of the file system includes a superblock, the superblock including journal index node content, universal unique identifier data, version information, and block size, and wherein the disk layout comprises a journal file and the data, distinct from the superblock, wherein the journal file comprises information capturing a most recent checkpoint from an operation log of the file system, and the data stores content of individual files within the file system;
- pre-allocating one or more chunks of storage space for the data into hierarchical lavers comprising an L0 layer indicating a status of each allocation unit (AU) in the chunks of storage space, and a second layer L1 that groups bits in the L0 layer to a group size that comprises the same number of bits as a CPU cache line size;
- writing the data into the one or more chunks of storage space without requiring a corresponding update to metadata;
- persisting the data;
- and
- persisting logs separately from the persisted data.
A system comprising:
- at least one processor;
- a key-value store;
- and
- at least one computer-readable medium containing instructions which, when executed, cause the at least one processor to perform operations comprising:
- maintaining separation of data and logs within a file system, wherein a disk layout of the file system includes a superblock, the superblock including journal index node content, universal unique identifier data, version information, and block size, and wherein the disk layout comprises a journal file and the data, distinct from the superblock, wherein the journal file comprises information capturing a most recent checkpoint from an operation log of the file system, and the data stores content of individual files within the file system;
- pre-allocating one or more chunks of storage space for the data into hierarchical layers comprising an L0 layer indicating a status of each allocation unit (AU) in the chunks of storage space, and a second layer L1 that groups bits in the L0 layer to a group size that comprises the same number of bits as a CPU cache line size;
- writing the data into the one or more chunks of storage space without requiring a corresponding update to metadata;
- persisting the data;
- and
- persisting logs separately from the persisted data.
A non-transitory computer-readable medium containing instructions that, when executed by a processor, direct establishment and operation of a file system, the file system configured to perform append operations, the file system configured for use with a key-value store, wherein the file system comprises a file manager configured to:
- maintain separation of data and logs within the file system, wherein a disk layout of the file system includes a superblock, the superblock including journal index node content, universal unique identifier data, version information, and block size, and wherein the disk layout comprises a journal file and the data, distinct from the superblock, wherein the journal file comprises information capturing a most recent checkpoint from an operation log of the file system, and the data stores content of individual files within the file system;
- pre-allocate one or more chunks of storage space for the data into hierarchical layers comprising an L0 layer indicating a status of each allocation unit (AU) in the chunks of storage space, and a second layer L1 that groups bits in the L0 layer to a group size that comprises the same number of bits as a CPU cache line size;
- write the data into the one or more chunks of storage space without requiring a corresponding update to metadata;
- persist the data;
- and
- persist logs separately from the persisted data.
Dependencies are inferred from the claim text.