Lockless distributed redundant storage and NVRAM caching of compressed data in a highly-distributed shared topology with direct memory access capable interconnect

A system for data storage, comprising:

• multiple servers, which are configured to communicate over a network with multiple multi-queue storage devices and with at least one storage controller, to store on the storage devices compressed data belonging to a user volume, to specify storage locations, in which the compressed data is stored on the storage devices, in a shared data structure that is shared and modified by the servers using remote direct memory access, and to coordinate access to the compressed data by the servers by querying the shared data structure, without executing code on a processor of the storage controller,
• wherein the servers are further configured to apply a background compression process, which compresses data that has already been stored on the storage devices, and updates the shared data structure so that the servers are able to access the compressed data concurrently and without executing code on the storage controller.