Methods and systems using multi-item containers in paged direct mapping key-value stores

A Key-Value data storage system, comprising:

• a storage device, the storage device being configured to store and enable access to user data via a Key-Value system;
• a memory;
• at least one processor;
• a plurality of processes spawned by the processor, the plurality of processes comprising processing logic to:
• iteratively:
• receive Key-Value data, the Key-Value data including a Key and a data Value;
• determine whether the Value data is a small Value data that is smaller than a predetermined threshold or a non-small Value data that is larger than or equal to the threshold;
• when the Value data is determined to be a small Value data, prepend a small Value header to the small Value data and temporarily store the small Value data and the prepended small Value header in a buffer in the memory, and
• when the Value data is determined to be a non-small Value data, compressing and prepend a non-small Value header to the non-small Value data, generate a single piece of metadata corresponding to the Key and to the non-small Value and send the non-small Value data and the prepended non-small Value header to a media allocator for storage to media in the storage device;
• determine to flush the buffer;
• compress, as a single compressed unit, the small Value data and the prepended small Value headers flushed from the buffer;
• generate and prepend a Multi-Item header to the single compressed unit to form a Multi-Item Container;
• generate a separate piece of metadata for each small Value data in the Multi-Item Container, and
• send the Multi-Item Container to the media allocator for storage of the Multi-Item Container to the media in the storage device.