Reduced write amplification for large-block data log
AssigneeVMware, Inc.
InventorsWenguang Wang, Vamsi Gunturu, Eric Knauft
Life of this patent
Estimating the term…
Abstract
Writing to a storage system with data striping includes storing blocks of data in local memory until one or more full-stripe write operations can be performed, thus reducing write amplification on the data striped storage system. Crash recovery information includes storing the data associated metadata to respective persistent storage devices. Metadata associated with data from several clients is combined into fixed-size data blocks and stored on the respective persistent storage device.
Claims20
A method comprising:
- receiving a write request comprising data to be written;
- storing the data to be written as a plurality of fixed-size data blocks in a memory bank of a main memory of a computer system;
- generating crash recovery information from the data to be written, including:
- persisting, without data striping, the plurality of fixed-size data blocks on a first block storage device;
- generating a plurality of metadata blocks from the data to be written;
- and
- persisting, without data striping, the plurality of metadata to a second block-based storage device;
- and
- subsequent to generating the crash recovery information, indicating an acknowledgement of the write request to the client;
- and
- writing data stored in the memory bank of the main memory to a data striped storage system only when enough data is accumulated from subsequent write requests that the accumulated data can be written to the data striped storage system in one or more full-stripe write operations.
A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computer device, cause the computer device to:
- receive a write request comprising data to be written;
- store the data to be written as a plurality of fixed-size data blocks in a memory bank of a main memory of the computer device;
- generate crash recovery information from the data to be written, including:
- persisting the plurality of fixed-size data blocks on a first block storage device;
- generating a plurality of metadata blocks from the data to be written;
- and
- persisting the plurality of metadata to a second block-based storage device;
- and
- subsequent to generating the crash recovery information, indicate an acknowledgement of the write request to the client;
- and
- write data stored in the memory bank of the main memory to a data striped storage system only when enough data is accumulated from subsequent write requests that the accumulated data can be written to the data striped storage system in one or more full-stripe write operations.
An apparatus comprising:
- one or more computer processors;
- a main memory;
- and
- a computer-readable storage medium comprising instructions for controlling the one or more computer processors to:
- receive a write request comprising data to be written;
- store the data to be written as a plurality of fixed-size data blocks in a memory bank of the main memory;
- generate crash recovery information from the data to be written, including:
- persisting the plurality of fixed-size data blocks on a first block storage device;
- generating a plurality of metadata blocks from the data to be written;
- and
- persisting the plurality of metadata to a second block-based storage device;
- and
- subsequent to generating the crash recovery information, indicate an acknowledgement of the write request to the client;
- and
- write data stored in the memory bank of the main memory to a data striped storage system only when enough data is accumulated from subsequent write requests that the accumulated data can be written to the data striped storage system in one or more full-stripe write operations.
Dependencies are inferred from the claim text.